DataStax Distribution of Apache Cassandra release notes

DataStax Distribution of Apache Cassandra release notes cover cluster requirements, upgrade guidance, security updates, changes and enhancements, issues, and resolved issues.

DataStax Distribution of Apache Cassandra™ (DDAC) release notes cover cluster requirements, upgrade guidance, components, changes and enhancements, issues, and resolved issues.
Note: Each point release includes a highlights and executive summary section to provide guidance and add visibility to important improvements.

Requirement for Uniform Licensing

All nodes in each cluster must be uniformly licensed to use the same subscription. For example, if a cluster contains 5 nodes, all 5 nodes within that cluster must be either DataStax Distribution of Apache Cassandra™, or all 5 nodes must be DataStax Enterprise. Mixing different subscriptions within a cluster is not permitted. The DataStax Advanced Workloads Pack may be added to any DataStax Enterprise (not DataStax Distribution of Apache Cassandra) cluster in an incremental fashion. For example, a 10-node DSE cluster may be extended to include 3 nodes of the Advanced Workloads Pack. “Cluster” means a collection of nodes running the software which communicate with one another using gossip. See Enterprise Terms.

Note: For third-party software, see DataStax Enterprise 5.1.x third-party software (not all entries apply to DDAC).

Before you upgrade

Upgrade advice Compatibility
Before you upgrade to a later major version, upgrade to the latest patch release (5.1.14) on your current version. Be sure to read the relevant upgrade documentation. DDAC supports patch upgrades starting with DDAC 5.1.11.
Check the compatibility page for your products. DDAC does not support OpsCenter or Studio.
See Upgrading DataStax drivers. DataStax Drivers: You may need to recompile your client application code.
Use DataStax Bulk Loader for loading and unloading data. Loads data into DDAC and unloads data from any Apache Cassandra™ 2.1 or later data source.

DataStax recommends upgrading to the latest patch release (5.1.14). Be sure to read the relevant upgrade documentation.

DDAC Compatibility

DDAC 5.1.14

Release notes for DataStax Distribution of Apache Cassandra 5.1.14.

Release notes for DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.14.
Important: DataStax recommends the latest patch release for most environments.

16 April 2019

DDAC 5.1.14 is compatible with Apache Cassandra™ 3.11 and adds production-certified changes to Cassandra.

5.1.14 DDAC database core highlights

  • Fixed anti-compaction transaction for atomicity and index building. (DB-3016)
  • Remedy deadlock during node startup when calculating disk boundaries. (DB-3028)

5.1.14 DDAC resolved issues

  • Native server Message.Dispatcher.Flusher task stalls under heavy load. (DB-1814)
  • Reference leak in SSTableRewriter in sstableupgrade when keepOriginals is true. (DB-2944)
  • Anti-compaction transaction causes temporary data loss. (DB-3016)
  • Check of two versions of metadata for a column fails on upgrade from DSE 5.0.x when type is not of same class. Loosen the check from CASSANDRA-13776 to prevent Trying to compare 2 different types ERROR on upgrades. (DB-3021)
  • Deadlock during node startup when calculating disk boundaries. (DB-3028)
  • Correct handling of dropped UDT columns in SSTables. (DB-3031)
  • Mishandling of frozen in complex nested types. (DB-3081)
  • cqlsh EXECUTE AS command does not work. (DB-3098)
  • Security: java-xmlbuilder is vulnerable to XML external entities (XXE). (DSP-13962)
  • Timestamp PK routing on solr_query fails. (DSP-18223)
  • Leak in BulkTableWriter. (DSP-18513)
  • Select Hadoop libraries

    Built-in Hadoop and Bring-Your-Own-Hadoop (BYOH) were deprecated in DataStax Enterprise (DSE) 5.0, and were removed in DSE 5.1. Hadoop removal from DSE 5.1 and later means that DSE does not allow for the startup of Hadoop services previously included in DSE, including MapReduce JobTracker and TaskTracker.

    However, DSE has supported built-in Spark since DSE 4.5 and Bring-Your-Own-Spark (BYOS) since DSE 5.0, and that support continues today. Because Spark depends on certain Hadoop libraries on the server and the client, DSE continues to ship with Hadoop libraries that are required for running Spark and BYOS.

    To view the included Hadoop libraries, see DataStax Enterprise 5.1.x third-party software.

Enhancements for DDAC 5.1.14

A list of DataStax Distribution of Apache Cassandra 5.1.14 enhancements to Apache Cassandra™ 3.11.

DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.14 includes this production-certified change and all enhancements from previous DDAC releases. (For updates, see CHANGES.txt.)

  • Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser (CASSANDRA-14781)

General upgrade advice for DDAC 5.1.14

General upgrade advice for DataStax Distribution of Apache Cassandra 5.1.14.

Carefully review all planning and upgrade documentation in the Upgrade Guide guide. This general advice applies to the database upgrade and does not replace the upgrade documentation.

PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.


GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.


DSE 5.1.12
==========

Operations
---------
   - A new property cassandra.range_tombstone_bound_check_chance was added to check for bad range tombstone on a
     percentage of queries.  The default is 0.01 (can be set in range of 0.0 - 1.0 if you wish to adjust).

   - Added "-Ddse.consistent_replace" startup option to support consistent replace by running background repair with
     streaming. Without consistent replace, node replacement only streams one copy of data which may be stale with 
     respect to QUORUM/LOCAL_QUORUM.
     When replacing a down node with "-Ddse.consistent_replace=QUORUM", the replacement node will coordinate repair
     among a quorum of replicas concurrently with replace streaming (or a local quorum of replicas when LOCAL_QUORUM is
     used). Repair will transfer the differences to the replacement node, ensuring it's consistent with other replicas
     when the replacement process is finished, assuming data is inserted with QUORUM/LOCAL_QUORUM.
     Available options: "ONE"(default), "LOCAL_QUORUM", "QUORUM".
     Startup option "-Ddse.consistent_replace.whitelist=<KEYSPACE>,<KEYSPACE.TABLE>,..." can be used to specify
     keyspaces or tables for consistent replace. By default, all keyspaces will be consistent replaced. Specifying
     keyspace without tables means all tables of the keyspace will be consistent replaced.


3.11.4
======

Upgrading
---------
    - The order of static columns in SELECT * has been fixed to match that of 2.0 and 2.1 - they are now sorted
      alphabetically again, by their name, just like regular columns are. If you use prepared statements and
      SELECT * queries, and have both simple and collection static columns in those tables, and are upgrading from an
      earlier 3.0 version, then you might be affected by this change. Please see CASSANDRA-14638 for details.


DSE 5.1.7
==========

Upgrading
---------
    - Automatic fallback of GossipingPropertyFileSnitch to PropertyFileSnitch (cassandra-topology.properties) is
      disabled by default and can be enabled via the -Dcassandra.gpfs.enable_pfs_compatibility_mode=true startup
      flag.

DSE 5.1.6
=========


3.11.3
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 (3.0.X series) or 3.11.1 (3.11.X series) and
      later that have performed range movements (join, decommission, move, etc), should run repair
      on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251
      is correctly propagated to all replicas.
    - Changes to bloom_filter_fp_chance will no longer take effect on existing sstables when the
      node is restarted. Only compactions/upgradesstables regenerates bloom filters and Summaries
      sstable components. See CASSANDRA-11163

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and read_repair_chance
      table options have been deprecated, and will be removed entirely in 4.0. See CASSANDRA-13910
      for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - seed_gossip_probability setting was added to cassandra.yaml.  This setting will pick the percentage of times
      gossip messages are sent to a seed.  This improves the time it takes for gossip changes to
      propagate across the cluster. Defaults to 100% (1.0)
    - Upgrades from DSE 5.0 might have produced unnecessary schema migrations while
      there was at least one DSE 5.0 node in the cluster. It is therefore highly
      recommended to upgrade from DSE 5.0 to at least DSE 5.1.6. The root cause of
      this schema mismatch was a difference in the way how schema digests were computed
      in DSE 5.0 and DSE 5.1. To mitigate this issue, DSE 5.1.6 and newer announce
      DSE 5.0 compatible digests as long as there is at least one DSE 5.0 node in the
      cluster. Once all nodes have been upgraded, the "real" schema version will be
      announced. Note: this fix is only necessary in DSE 5.1 and therefore only applies
      to DSE 5.1. (DB-1477)
    - DSE is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details
    - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
      set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.
    - Improved gossip settling added. On startup DSE waits till all nodes are seen before fully joining the cluster.
      This improves latency spikes when restarting nodes.
    - LeveledCompactionStrategy SSTables will keep their existing level on nodetool refresh, nodetool move,
      and nodetool decommission.

Operations
----------
    - It is now possible to ALTER system_distributed tables
    - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
      The command must be executed on the node where the rebuild operation is running. Streams
      may continue until they finish or timeout.
    - Only MODIFY permission on base is required to update table with MV, internally it reads base
      data and generates updates to MV.

Metrics
-------
   - New storage metrics were added:
     * TotalHintsReplayed: how many hints were successfully replayed on the _target_ node.
     * HintsOnDisk: how many hints are currently persistent on disk on this node. Metric is updated
       for the amount of hints contained in the hints file when hints file is written or removed.
       Values is restored on node startup.

New features
------------
   -  Statistics file component was added to Hint Store in order to provide information about
      amount of hints contained in the hints file withouot replaying it. Stats component is
      completely backwad-compatible; hint files withouot this component will not be counted.
      All new hint files will be created with this component. See DB-853 for more details.


DSE 5.0.10
==========

Upgrading
---------
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on DSE 5.0.10. Upgrading to DSE 5.1 requires at least DSE 5.1.3.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.


3.0.17
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 or later that have performed range movements (join, decommission, move, etc),
      should run repair on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251 is correctly
      propagated to all replicas.

3.0.16
=====

Upgrading
---------
   - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
   - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
     rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
     are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
     options are used. See CASSANDRA-13006 for more details.

3.0.15
=====

Upgrading
---------
   - Nothing specific to this release, but please see previous upgrading sections,
     especially if you are upgrading from 2.2.

Compact Storage
---------------
    - Starting version 4.0, Thrift and COMPACT STORAGE is no longer supported.
      'ALTER ... DROP COMPACT STORAGE' statement makes Compact Tables CQL-compatible,
      exposing internal structure of Thrift/Compact Tables. You can find more details
      on exposed internal structure under:
      http://cassandra.apache.org/doc/latest/cql/appendices.html#appendix-c-dropping-compact-storage

      For uninterrupted cluster upgrades, drivers now support 'NO_COMPACT' startup option.
      Supplying this flag will have same effect as 'DROP COMPACT STORAGE', but only for the
      current connection.

      In order to upgrade, clients supporting a non-compact schema view can be rolled out
      gradually. When all the clients are updated 'ALTER ... DROP COMPACT STORAGE' can be
      executed. After dropping compact storage, ’NO_COMPACT' option will have no effect
      after that.

DSE 5.1.3
=========

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 5.1.3. Note that upgrades from DSE 5.0 series since 5.0.10 to DSE
     versions before 5.1.3 are not recommended.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.

Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.11.2
======

Upgrading
---------
    - Nothing specific to this release, but please see previous upgrading sections.

3.11.1
======

Upgrading
---------
    - Creating Materialized View with filtering on non-primary-key base column
      (added in CASSANDRA-10368) is disabled, because the liveness of view row
      is depending on multiple filtered base non-key columns and base non-key
      column used in view primary-key. This semantic cannot be supported without
      storage format change, see CASSANDRA-13826. For append-only use case, you
      may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"

3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - ALTER TABLE (ADD/DROP COLUMN) operations concurrent with a read might
     result into data corruption (see CASSANDRA-13004 for more details).
     Fixing this bug required a messaging protocol version bump. By default,
     Cassandra 3.11 will use 3014 version for messaging.

     Since Schema Migrations rely the on exact messaging protocol version
     match between nodes, if you need schema changes during the upgrade
     process, you have to start your nodes with `-Dcassandra.force_3_0_protocol_version=true`
     first, in order to temporarily force a backwards compatible protocol.
     After the whole cluster is upgraded to 3.11, do a rolling
     restart of the cluster without setting that flag.

     3.11 nodes with and withouot the flag set will be able to do schema
     migrations with other 3.x and 3.0.x releases.

     While running the cluster with the flag set to true on 3.11 (in
     compatibility mode), avoid adding or removing any columns to/from
     existing tables.

     If your cluster can do without schema migrations during the upgrade
     time, just start the cluster normally without setting aforementioned
     flag.

     If you are upgrading from 3.0.14+ (of 3.0.x branch), you do not have
     to set an flag while upgrading to ensure schema migrations.
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).


3.0
===

New features
------------
   - EACH_QUORUM is now a supported consistency level for read requests.
   - Support for IN restrictions on any partition key component or clustering key
     as well as support for EQ and IN multicolumn restrictions has been added to
     UPDATE and DELETE statement.
   - Support for single-column and multi-colum slice restrictions (>, >=, <= and <)
     has been added to DELETE statements
   - nodetool rebuild_index accepts the index argument without
     the redundant table name
   - Materialized Views, which allow for server-side denormalization, is now
     available. Materialized views provide an alternative to secondary indexes
     for non-primary key queries, and perform much better for indexing high
     cardinality columns.
     See http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
   - Hinted handoff has been completely rewritten. Hints are now stored in flat
     files, with less overhead for storage and more efficient dispatch.
     See CASSANDRA-6230 for full details.
   - Option to not purge unrepaired tombstones. To avoid users having data resurrected
     if repair has not been run within gc_grace_seconds, an option has been added to
     only allow tombstones from repaired sstables to be purged. To enable, set the
     compaction option 'only_purge_repaired_tombstones':true but keep in mind that if
     you do not run repair for a long time, you will keep all tombstones around which
     can cause other problems.
   - Enabled warning on GC taking longer than 1000ms. See
     cassandra.yaml:gc_warn_threshold_in_ms

Upgrading
---------
   - Clients must use the native protocol version 3 when upgrading from 2.2.X as
     the native protocol version 4 is not compatible between 2.2.X and 3.Y. See
     https://www.mail-archive.com/user@cassandra.apache.org/msg45381.html for details.
   - A new argument of type InetAdress has been added to IAuthenticator::newSaslNegotiator,
     representing the IP address of the client attempting authentication. It will be a breaking
     change for any custom implementations.
   - token-generator tool has been removed.
   - Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal to 2.1.9,
     or Cassandra 2.2 versions greater or equal to 2.2.2. Upgrade from Cassandra 2.0 and
     older versions is not supported.
   - The 'memtable_allocation_type: offheap_objects' option has been removed. It should
     be re-introduced in a future release and you can follow CASSANDRA-9472 to know more.
   - Configuration parameter memory_allocator in cassandra.yaml has been removed.
   - The native protocol versions 1 and 2 are not supported anymore.
   - Max mutation size is now configurable via max_mutation_size_in_kb setting in
     cassandra.yaml; the default is half the size commitlog_segment_size_in_mb * 1024.
   - 3.0 requires Java 8u40 or later.
   - Garbage collection options were moved from cassandra-env to jvm.options file.
   - New transaction log files have been introduced to replace the compactions_in_progress
     system table, temporary file markers (tmp and tmplink) and sstable ancerstors.
     Therefore, compaction metadata no longer contains ancestors. Transaction log files
     list sstable descriptors involved in compactions and other operations such as flushing
     and streaming. Use the sstableutil tool to list any sstable files currently involved
     in operations not yet completed, which previously would have been marked as temporary.
     A transaction log file contains one sstable per line, with the prefix "add:" or "remove:".
     They also contain a special line "commit", only inserted at the end when the transaction
     is committed. On startup we use these files to cleanup any partial transactions that were
     in progress when the process exited. If the commit line is found, we keep new sstables
     (those with the "add" prefix) and delete the old sstables (those with the "remove" prefix),
     vice-versa if the commit line is missing. Should you lose or delete these log files,
     both old and new sstable files will be kept as live files, which will result in duplicated
     sstables. These files are protected by incremental checksums so you should not manually
     edit them. When restoring a full backup or moving sstable files, you should clean-up
     any left over transactions and their temporary files first. You can use this command:
      ===> sstableutil -c ks table
     See CASSANDRA-7066 for full details.
   - New write stages have been added for batchlog and materialized view mutations
     you can set their size in cassandra.yaml
   - User defined functions are now executed in a sandbox.
     To use UDFs and UDAs, you have to enable them in cassandra.yaml.
   - New SSTable version 'la' with improved bloom-filter false-positive handling
     compared to previous version 'ka' used in 2.2 and 2.1. Running sstableupgrade
     is not necessary but recommended.
   - Before upgrading to 3.0, make sure that your cluster is in complete agreement
     (schema versions outputted by `nodetool describecluster` are all the same).
   - Schema metadata is now stored in the new `system_schema` keyspace, and
     legacy `system.schema_*` tables are now gone; see CASSANDRA-6717 for details.
   - Pig's support has been removed.
   - Hadoop BulkOutputFormat and BulkRecordWriter have been removed; use
     CqlBulkOutputFormat and CqlBulkRecordWriter instead.
   - Hadoop ColumnFamilyInputFormat and ColumnFamilyOutputFormat have been removed;
     use CqlInputFormat and CqlOutputFormat instead.
   - Hadoop ColumnFamilyRecordReader and ColumnFamilyRecordWriter have been removed;
     use CqlRecordReader and CqlRecordWriter instead.
   - hinted_handoff_enabled in cassandra.yaml no longer supports a list of data centers.
     To specify a list of excluded data centers when hinted_handoff_enabled is set to true,
     use hinted_handoff_disabled_datacenters, see CASSANDRA-9035 for details.
   - The `sstable_compression` and `chunk_length_kb` compression options have been deprecated.
     The new options are `class` and `chunk_length_in_kb`. Disabling compression should now
     be done by setting the new option `enabled` to `false`.
   - The compression option `crc_check_chance` became a top-level table option, but is currently
     enforced only against tables with enabled compression.
   - Only map syntax is now allowed for caching options. ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax
     has been deprecated since 2.1.0 and is being removed in 3.0.0.
   - The 'index_interval' option for 'CREATE TABLE' statements, which has been deprecated
     since 2.1 and replaced with the 'min_index_interval' and 'max_index_interval' options,
     has now been removed.
   - Batchlog entries are now stored in a new table - system.batches.
     The old one has been deprecated.
   - JMX methods set/getCompactionStrategyClass have been removed, use
     set/getCompactionParameters or set/getCompactionParametersJson instead.
   - SizeTieredCompactionStrategy parameter cold_reads_to_omit has been removed.
   - The secondary index API has been comprehensively reworked. This will be a breaking
     change for any custom index implementations, which should now look to implement
     the new org.apache.cassandra.index.Index interface. New syntax has been added to create
     and query row-based indexes, which are not explicitly linked to a single column in the
     base table.

DDAC 5.1.13

Release notes for DataStax Distribution of Apache Cassandra 5.1.13.

Release notes for DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.13.
Important: DataStax recommends the latest patch release for most environments.

27 February 2019

DDAC 5.1.13 is compatible with Apache Cassandra™ 3.11 and includes all production-certified changes from previous releases.

Enhancements for DDAC 5.1.13

A list of DataStax Distribution of Apache Cassandra 5.1.13 enhancements to Apache Cassandra™ 3.11.

DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.13 includes all enhancements from previous DDAC releases. (For updates, see CHANGES.txt.)

General upgrade advice for DSE 5.1.13

General upgrade advice for DataStax Enterprise 5.1.13.

Carefully review all planning and upgrade documentation in the Upgrade Guide guide. This general advice applies to the database upgrade and does not replace the upgrade documentation.

PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
------------------------------------------------------------------
QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
SocketServer and ServerSocketReceiver components.

Logback has not been upgraded to avoid breaking deployments and customizations
based on older versions. If you are using vulnerable components you will need
to upgrade to a newer version of Logback or stop using the vulnerable components.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.


DSE 5.1.13
==========

Upgrading
---------
   - Sstables for tables using with a frozen UDT written by DSE 5.0 or C* 3.0 appear as corrupted.

     Background: The serialization-header in the -Statistics.db sstable component contains the type information
     of the table columns. C* 3.0 (and DSE 5.0) write incorrect type information for frozen UDTs by omitting the
     "frozen" information. Non-frozen UDTs were introduced by CASSANDRA-7423 in C* 3.6. Since then, the missing
     "frozen" information leads to deserialization issues that result in CorruptSSTableExceptions, potentially other
     exceptions as well.

     As a mitigation, the sstable serialization-headers are rewritten to contain the missing "frozen" information for
     UDTs once, when an upgrade from DSE 5.0 / C* 3.0 is detected. This migration does not touch snapshots or backups.

     The sstablescrub tool now performs a check of the sstable serialization-header against the schema. A mismatch of
     the types in the serialization-header and the schema will cause sstablescrub to error out and stop by default.
     See the new `-e` option. `-e off` disables the new validation code. `-e fix` or `-e fix-only`, e.g.
     `sstablescrub -e fix keyspace table`, will validate the serialization-header, rewrite the non-frozen UDTs
     in the serialzation-header to frozen UDTs, if that matches the schema, and continue with scrub.
     See `sstablescrub -h`.
     (DB-2954)


DSE 5.1.12
==========

Operations
---------
   - A new property cassandra.range_tombstone_bound_check_chance was added to check for bad range tombstone on a
     percentage of queries.  The default is 0.01 (can be set in range of 0.0 - 1.0 if you wish to adjust).


   - Added "-Ddse.consistent_replace" startup option to support consistent replace by running background repair with
     streaming. Without consistent replace, node replacement only streams one copy of data which may be stale with
     respect to QUORUM/LOCAL_QUORUM.
     When replacing a down node with "-Ddse.consistent_replace=QUORUM", the replacement node will coordinate repair
     among a quorum of replicas concurrently with replace streaming (or a local quorum of replicas when LOCAL_QUORUM is
     used). Repair will transfer the differences to the replacement node, ensuring it's consistent with other replicas
     when the replacement process is finished, assuming data is inserted with QUORUM/LOCAL_QUORUM.
     Available options: "ONE"(default), "LOCAL_QUORUM", "QUORUM".
     Startup option "-Ddse.consistent_replace.whitelist=<KEYSPACE>,<KEYSPACE.TABLE>,..." can be used to specify
     keyspaces or tables for consistent replace. By default, all keyspaces will be consistent replaced. Specifying
     keyspace without tables means all tables of the keyspace will be consistent replaced.


3.11.4
======

Upgrading
---------
    - The order of static columns in SELECT * has been fixed to match that of 2.0 and 2.1 - they are now sorted
      alphabetically again, by their name, just like regular columns are. If you use prepared statements and
      SELECT * queries, and have both simple and collection static columns in those tables, and are upgrading from an
      earlier 3.0 version, then you might be affected by this change. Please see CASSANDRA-14638 for details.


DSE 5.1.7
==========

Upgrading
---------
    - Automatic fallback of GossipingPropertyFileSnitch to PropertyFileSnitch (cassandra-topology.properties) is
      disabled by default and can be enabled via the -Dcassandra.gpfs.enable_pfs_compatibility_mode=true startup
      flag.

DSE 5.1.6
=========


3.11.3
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 (3.0.X series) or 3.11.1 (3.11.X series) and
      later that have performed range movements (join, decommission, move, etc), should run repair
      on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251
      is correctly propagated to all replicas.
    - Changes to bloom_filter_fp_chance will no longer take effect on existing sstables when the
      node is restarted. Only compactions/upgradesstables regenerates bloom filters and Summaries
      sstable components. See CASSANDRA-11163

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and read_repair_chance
      table options have been deprecated, and will be removed entirely in 4.0. See CASSANDRA-13910
      for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - seed_gossip_probability setting was added to cassandra.yaml.  This setting will pick the percentage of times
      gossip messages are sent to a seed.  This improves the time it takes for gossip changes to
      propagate across the cluster. Defaults to 100% (1.0)
    - Upgrades from DSE 5.0 might have produced unnecessary schema migrations while
      there was at least one DSE 5.0 node in the cluster. It is therefore highly
      recommended to upgrade from DSE 5.0 to at least DSE 5.1.6. The root cause of
      this schema mismatch was a difference in the way how schema digests were computed
      in DSE 5.0 and DSE 5.1. To mitigate this issue, DSE 5.1.6 and newer announce
      DSE 5.0 compatible digests as long as there is at least one DSE 5.0 node in the
      cluster. Once all nodes have been upgraded, the "real" schema version will be
      announced. Note: this fix is only necessary in DSE 5.1 and therefore only applies
      to DSE 5.1. (DB-1477)
    - DSE is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details
    - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
      set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.
    - Improved gossip settling added. On startup DSE waits till all nodes are seen before fully joining the cluster.
      This improves latency spikes when restarting nodes.
    - LeveledCompactionStrategy SSTables will keep their existing level on nodetool refresh, nodetool move,
      and nodetool decommission.

Operations
----------
    - It is now possible to ALTER system_distributed tables
    - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
      The command must be executed on the node where the rebuild operation is running. Streams
      may continue until they finish or timeout.
    - Only MODIFY permission on base is required to update table with MV, internally it reads base
      data and generates updates to MV.

Metrics
-------
   - New storage metrics were added:
     * TotalHintsReplayed: how many hints were successfully replayed on the _target_ node.
     * HintsOnDisk: how many hints are currently persistent on disk on this node. Metric is updated
       for the amount of hints contained in the hints file when hints file is written or removed.
       Values is restored on node startup.

New features
------------
   -  Statistics file component was added to Hint Store in order to provide information about
      amount of hints contained in the hints file withouot replaying it. Stats component is
      completely backwad-compatible; hint files withouot this component will not be counted.
      All new hint files will be created with this component. See DB-853 for more details.


DSE 5.0.10
==========

Upgrading
---------
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on DSE 5.0.10. Upgrading to DSE 5.1 requires at least DSE 5.1.3.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.


3.0.17
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 or later that have performed range movements (join, decommission, move, etc),
      should run repair on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251 is correctly
      propagated to all replicas.

3.0.16
=====

Upgrading
---------
   - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
   - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
     rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
     are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
     options are used. See CASSANDRA-13006 for more details.

3.0.15
=====

Upgrading
---------
   - Nothing specific to this release, but please see previous upgrading sections,
     especially if you are upgrading from 2.2.

Compact Storage
---------------
    - Starting version 4.0, Thrift and COMPACT STORAGE is no longer supported.
      'ALTER ... DROP COMPACT STORAGE' statement makes Compact Tables CQL-compatible,
      exposing internal structure of Thrift/Compact Tables. You can find more details
      on exposed internal structure under:
      http://cassandra.apache.org/doc/latest/cql/appendices.html#appendix-c-dropping-compact-storage

      For uninterrupted cluster upgrades, drivers now support 'NO_COMPACT' startup option.
      Supplying this flag will have same effect as 'DROP COMPACT STORAGE', but only for the
      current connection.

      In order to upgrade, clients supporting a non-compact schema view can be rolled out
      gradually. When all the clients are updated 'ALTER ... DROP COMPACT STORAGE' can be
      executed. After dropping compact storage, ’NO_COMPACT' option will have no effect
      after that.

DSE 5.1.3
=========

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 5.1.3. Note that upgrades from DSE 5.0 series since 5.0.10 to DSE
     versions before 5.1.3 are not recommended.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.

Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.11.2
======

Upgrading
---------
    - Nothing specific to this release, but please see previous upgrading sections.

3.11.1
======

Upgrading
---------
    - Creating Materialized View with filtering on non-primary-key base column
      (added in CASSANDRA-10368) is disabled, because the liveness of view row
      is depending on multiple filtered base non-key columns and base non-key
      column used in view primary-key. This semantic cannot be supported without
      storage format change, see CASSANDRA-13826. For append-only use case, you
      may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"

3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - ALTER TABLE (ADD/DROP COLUMN) operations concurrent with a read might
     result into data corruption (see CASSANDRA-13004 for more details).
     Fixing this bug required a messaging protocol version bump. By default,
     Cassandra 3.11 will use 3014 version for messaging.

     Since Schema Migrations rely the on exact messaging protocol version
     match between nodes, if you need schema changes during the upgrade
     process, you have to start your nodes with `-Dcassandra.force_3_0_protocol_version=true`
     first, in order to temporarily force a backwards compatible protocol.
     After the whole cluster is upgraded to 3.11, do a rolling
     restart of the cluster without setting that flag.

     3.11 nodes with and withouot the flag set will be able to do schema
     migrations with other 3.x and 3.0.x releases.

     While running the cluster with the flag set to true on 3.11 (in
     compatibility mode), avoid adding or removing any columns to/from
     existing tables.

     If your cluster can do without schema migrations during the upgrade
     time, just start the cluster normally without setting aforementioned
     flag.

     If you are upgrading from 3.0.14+ (of 3.0.x branch), you do not have
     to set an flag while upgrading to ensure schema migrations.
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).


3.0
===

New features
------------
   - EACH_QUORUM is now a supported consistency level for read requests.
   - Support for IN restrictions on any partition key component or clustering key
     as well as support for EQ and IN multicolumn restrictions has been added to
     UPDATE and DELETE statement.
   - Support for single-column and multi-colum slice restrictions (>, >=, <= and <)
     has been added to DELETE statements
   - nodetool rebuild_index accepts the index argument without
     the redundant table name
   - Materialized Views, which allow for server-side denormalization, is now
     available. Materialized views provide an alternative to secondary indexes
     for non-primary key queries, and perform much better for indexing high
     cardinality columns.
     See http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
   - Hinted handoff has been completely rewritten. Hints are now stored in flat
     files, with less overhead for storage and more efficient dispatch.
     See CASSANDRA-6230 for full details.
   - Option to not purge unrepaired tombstones. To avoid users having data resurrected
     if repair has not been run within gc_grace_seconds, an option has been added to
     only allow tombstones from repaired sstables to be purged. To enable, set the
     compaction option 'only_purge_repaired_tombstones':true but keep in mind that if
     you do not run repair for a long time, you will keep all tombstones around which
     can cause other problems.
   - Enabled warning on GC taking longer than 1000ms. See
     cassandra.yaml:gc_warn_threshold_in_ms

Upgrading
---------
   - Clients must use the native protocol version 3 when upgrading from 2.2.X as
     the native protocol version 4 is not compatible between 2.2.X and 3.Y. See
     https://www.mail-archive.com/user@cassandra.apache.org/msg45381.html for details.
   - A new argument of type InetAdress has been added to IAuthenticator::newSaslNegotiator,
     representing the IP address of the client attempting authentication. It will be a breaking
     change for any custom implementations.
   - token-generator tool has been removed.
   - Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal to 2.1.9,
     or Cassandra 2.2 versions greater or equal to 2.2.2. Upgrade from Cassandra 2.0 and
     older versions is not supported.
   - The 'memtable_allocation_type: offheap_objects' option has been removed. It should
     be re-introduced in a future release and you can follow CASSANDRA-9472 to know more.
   - Configuration parameter memory_allocator in cassandra.yaml has been removed.
   - The native protocol versions 1 and 2 are not supported anymore.
   - Max mutation size is now configurable via max_mutation_size_in_kb setting in
     cassandra.yaml; the default is half the size commitlog_segment_size_in_mb * 1024.
   - 3.0 requires Java 8u40 or later.
   - Garbage collection options were moved from cassandra-env to jvm.options file.
   - New transaction log files have been introduced to replace the compactions_in_progress
     system table, temporary file markers (tmp and tmplink) and sstable ancerstors.
     Therefore, compaction metadata no longer contains ancestors. Transaction log files
     list sstable descriptors involved in compactions and other operations such as flushing
     and streaming. Use the sstableutil tool to list any sstable files currently involved
     in operations not yet completed, which previously would have been marked as temporary.
     A transaction log file contains one sstable per line, with the prefix "add:" or "remove:".
     They also contain a special line "commit", only inserted at the end when the transaction
     is committed. On startup we use these files to cleanup any partial transactions that were
     in progress when the process exited. If the commit line is found, we keep new sstables
     (those with the "add" prefix) and delete the old sstables (those with the "remove" prefix),
     vice-versa if the commit line is missing. Should you lose or delete these log files,
     both old and new sstable files will be kept as live files, which will result in duplicated
     sstables. These files are protected by incremental checksums so you should not manually
     edit them. When restoring a full backup or moving sstable files, you should clean-up
     any left over transactions and their temporary files first. You can use this command:
      ===> sstableutil -c ks table
     See CASSANDRA-7066 for full details.
   - New write stages have been added for batchlog and materialized view mutations
     you can set their size in cassandra.yaml
   - User defined functions are now executed in a sandbox.
     To use UDFs and UDAs, you have to enable them in cassandra.yaml.
   - New SSTable version 'la' with improved bloom-filter false-positive handling
     compared to previous version 'ka' used in 2.2 and 2.1. Running sstableupgrade
     is not necessary but recommended.
   - Before upgrading to 3.0, make sure that your cluster is in complete agreement
     (schema versions outputted by `nodetool describecluster` are all the same).
   - Schema metadata is now stored in the new `system_schema` keyspace, and
     legacy `system.schema_*` tables are now gone; see CASSANDRA-6717 for details.
   - Pig's support has been removed.
   - Hadoop BulkOutputFormat and BulkRecordWriter have been removed; use
     CqlBulkOutputFormat and CqlBulkRecordWriter instead.
   - Hadoop ColumnFamilyInputFormat and ColumnFamilyOutputFormat have been removed;
     use CqlInputFormat and CqlOutputFormat instead.
   - Hadoop ColumnFamilyRecordReader and ColumnFamilyRecordWriter have been removed;
     use CqlRecordReader and CqlRecordWriter instead.
   - hinted_handoff_enabled in cassandra.yaml no longer supports a list of data centers.
     To specify a list of excluded data centers when hinted_handoff_enabled is set to true,
     use hinted_handoff_disabled_datacenters, see CASSANDRA-9035 for details.
   - The `sstable_compression` and `chunk_length_kb` compression options have been deprecated.
     The new options are `class` and `chunk_length_in_kb`. Disabling compression should now
     be done by setting the new option `enabled` to `false`.
   - The compression option `crc_check_chance` became a top-level table option, but is currently
     enforced only against tables with enabled compression.
   - Only map syntax is now allowed for caching options. ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax
     has been deprecated since 2.1.0 and is being removed in 3.0.0.
   - The 'index_interval' option for 'CREATE TABLE' statements, which has been deprecated
     since 2.1 and replaced with the 'min_index_interval' and 'max_index_interval' options,
     has now been removed.
   - Batchlog entries are now stored in a new table - system.batches.
     The old one has been deprecated.
   - JMX methods set/getCompactionStrategyClass have been removed, use
     set/getCompactionParameters or set/getCompactionParametersJson instead.
   - SizeTieredCompactionStrategy parameter cold_reads_to_omit has been removed.
   - The secondary index API has been comprehensively reworked. This will be a breaking
     change for any custom index implementations, which should now look to implement
     the new org.apache.cassandra.index.Index interface. New syntax has been added to create
     and query row-based indexes, which are not explicitly linked to a single column in the
     base table.


2.2.4
=====

Deprecation
-----------
    - Pig support has been deprecated, and will be removed in 3.0.
      Please see CASSANDRA-10542 for more details.
    - Configuration parameter memory_allocator in cassandra.yaml has been deprecated
      and will be removed in 3.0.0. As mentioned below for 2.2.0, jemalloc is
      automatically preloaded on Unix platforms.

Operations
----------
    - Switching data center or racks is no longer an allowed operation on a node
      which has data. Instead, the node will need to be decommissioned and
      rebootstrapped. If moving from the SimpleSnitch, make sure that the data
      center and rack containing all current nodes is named "datacenter1" and
      "rack1". To override this behaviour use -Dcassandra.ignore_rack=true and/or
      -Dcassandra.ignore_dc=true.
    - Reloading the configuration file of GossipingPropertyFileSnitch has been disabled.

Upgrading
---------
    - The default for the inter-DC stream throughput setting
      (inter_dc_stream_throughput_outbound_megabits_per_sec in cassandra.yaml) is
      the same than the one for intra-DC one (200Mbps) instead of being unlimited.
      Having it unlimited was never intended and was a bug.

New features
------------
    - Time windows in DTCS are now limited to 1 day by default to be able to
      handle bootstrap and repair in a better way. To get the old behaviour,
      increase max_window_size_seconds.
    - DTCS option max_sstable_age_days is now deprecated and defaults to 1000 days.
    - Native protocol server now allows both SSL and non-SSL connections on
      the same port.

2.2.3
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.2 if you are upgrading
      from a previous version.


2.2.2
=====

Changed Defaults
----------------
   - commitlog_total_space_in_mb will use the smaller of 8192, and 1/4
     of the total space of the commitlog volume. (Before: always used
     8192)
   - The following INFO logs were reduced to DEBUG level and will now show
     on debug.log instead of system.log:
      - Memtable flushing actions
      - Commit log replayed files
      - Compacted sstables
      - SStable opening (SSTableReader)

New features
------------
   - Custom QueryHandlers can retrieve the column specifications for the bound
     variables from QueryOptions by using the hasColumnSpecifications()
     and getColumnSpecifications() methods.
   - A new default assynchronous log appender debug.log was created in addition
     to  the system.log appender in order to provide more detailed log debugging.
     In order to disable debug logging, you must comment-out the ASYNCDEBUGLOG
     appender on conf/logback.xml. See CASSANDRA-10241 for more information.


2.2.1
=====

New features
------------
   - COUNT(*) and COUNT(1) can be selected with other columns or functions


2.2
===

Upgrading
---------
   - The authentication & authorization subsystems have been redesigned to
     support role based access control (RBAC), resulting in a change to the
     schema of the system_auth keyspace. See below for more detail.
     For systems already using the internal auth implementations, the process
     for converting existing data during a rolling upgrade is straightforward.
     As each node is restarted, it will attempt to convert any data in the
     legacy tables into the new schema. Until enough nodes to satisfy the
     replication strategy for the system_auth keyspace are upgraded and so have
     the new schema, this conversion will fail with the failure being reported
     in the system log.
     During the upgrade, Cassandra's internal auth classes will continue to use
     the legacy tables, so clients experience no disruption. Issuing DCL
     statements during an upgrade is not supported.
     Once all nodes are upgraded, an operator with superuser privileges should
     drop the legacy tables, system_auth.users, system_auth.credentials and
     system_auth.permissions. Doing so will prompt Cassandra to switch over to
     the new tables without requiring any further intervention.
     While the legacy tables are present a restarted node will re-run the data
     conversion and report the outcome so that operators can verify that it is
     safe to drop them.

New features
------------
   - The LIMIT clause applies now only to the number of rows returned to the user,
     not to the number of row queried. By consequence, queries using aggregates will not
     be impacted by the LIMIT clause anymore.
   - Very large batches will now be rejected (defaults to 50kb). This
     can be customized by modifying batch_size_fail_threshold_in_kb.
   - Selecting columns,scalar functions, UDT fields, writetime or ttl together
     with aggregated is now possible. The value returned for the columns,
     scalar functions, UDT fields, writetime and ttl will be the ones for
     the first row matching the query.
   - Windows is now a supported platform. Powershell execution for startup scripts
     is highly recommended and can be enabled via an administrator command-prompt
     with: 'powershell set-executionpolicy unrestricted'
   - It is now possible to do major compactions when using leveled compaction.
     Doing that will take all sstables and compact them out in levels. The
     levels will be non overlapping so doing this will still not be something
     you want to do very often since it might cause more compactions for a while.
     It is also possible to split output when doing a major compaction with
     STCS - files will be split in sizes 50%, 25%, 12.5% etc of the total size.
     This might be a bit better than old major compactions which created one big
     file on disk.
   - A new tool has been added bin/sstableverify that checks for errors/bitrot
     in all sstables.  Unlike scrub, this is a non-invasive tool.
   - Authentication & Authorization APIs have been updated to introduce
     roles. Roles and Permissions granted to them are inherited, supporting
     role based access control. The role concept supercedes that of users
     and CQL constructs such as CREATE USER are deprecated but retained for
     compatibility. The requirement to explicitly create Roles in Cassandra
     even when auth is handled by an external system has been removed, so
     authentication & authorization can be delegated to such systems in their
     entirety.
   - In addition to the above, Roles are also first class resources and can be the
     subject of permissions. Users (roles) can now be granted permissions on other
     roles, including CREATE, ALTER, DROP & AUTHORIZE, which removesthe need for
     superuser privileges in order to perform user/role management operations.
   - Creators of database resources (Keyspaces, Tables, Roles) are now automatically
     granted all permissions on them (if the IAuthorizer implementation supports
     this).
   - SSTable file name is changed. Now you don't have Keyspace/CF name
     in file name. Also, secondary index has its own directory under parent's
     directory.
   - Support for user-defined functions and user-defined aggregates have
     been added to CQL.
     ************************************************************************
     IMPORTANT NOTE: user-defined functions can be used to execute
     arbitrary and possibly evil code in Cassandra 2.2, and are
     therefore disabled by default.  To enable UDFs edit
     cassandra.yaml and set enable_user_defined_functions to true.

     CASSANDRA-9402 will add a security manager for UDFs in Cassandra
     3.0.  This will inherently be backwards-incompatible with any 2.2
     UDF that perform insecure operations such as opening a socket or
     writing to the filesystem.
     ************************************************************************
   - Row-cache is now fully off-heap.
   - jemalloc is now automatically preloaded and used on Linux and OS-X if
     installed.
   - Please ensure on Unix platforms that there is no libjnadispath.so
     installed which is accessible by Cassandra. Old versions of
     libjna packages (< 4.0.0) will cause problems - e.g. Debian Wheezy
     contains libjna versin 3.2.x.
   - The node now keeps up when streaming is failed during bootstrapping. You can
     use new `nodetool bootstrap resume` command to continue streaming after resolving
     an issue.
   - Protocol version 4 specifies that bind variables do not require having a
     value when executing a statement. Bind variables without a value are
     called 'unset'. The 'unset' bind variable is serialized as the int
     value '-2' without following bytes.
     In an EXECUTE or BATCH request an unset bind value does not modify the value and
     does not create a tombstone, an unset bind ttl is treated as 'unlimited',
     an unset bind timestamp is treated as 'now', an unset bind counter operation
     does not change the counter value.
     Unset tuple field, UDT field and map key are not allowed.
     In a QUERY request an unset limit is treated as 'unlimited'.
     Unset WHERE clauses with unset partition column, clustering column
     or index column are not allowed.
   - New `ByteType` (cql tinyint). 1-byte signed integer
   - New `ShortType` (cql smallint). 2-byte signed integer
   - New `SimpleDateType` (cql date). 4-byte unsigned integer
   - New `TimeType` (cql time). 8-byte long
   - The toDate(timeuuid), toTimestamp(timeuuid) and toUnixTimestamp(timeuuid) functions have been added to allow
     to convert from timeuuid into date type, timestamp type and bigint raw value.
     The functions unixTimestampOf(timeuuid) and dateOf(timeuuid) have been deprecated.
   - The toDate(timestamp) and toUnixTimestamp(timestamp) functions have been added to allow
     to convert from timestamp into date type and bigint raw value.
   - The toTimestamp(date) and toUnixTimestamp(date) functions have been added to allow
     to convert from date into timestamp type and bigint raw value.
   - SizeTieredCompactionStrategy parameter cold_reads_to_omit has been removed.
   - The default JVM flag -XX:+PerfDisableSharedMem will cause the following tools JVM
     to stop working: jps, jstack, jinfo, jmc, jcmd as well as 3rd party tools like Jolokia.
     If you wish to use these tools you can comment this flag out in cassandra-env.{sh,ps1}

Upgrading
---------
   - Thrift rpc is no longer being started by default.
     Set `start_rpc` parameter to `true` to enable it.
   - Pig's CqlStorage has been removed, use CqlNativeStorage instead
   - Pig's CassandraStorage has been deprecated. CassandraStorage
     should only be used against tables created via thrift.
     Use CqlNativeStorage for all other tables.
   - IAuthenticator been updated to remove responsibility for user/role
     maintenance and is now solely responsible for validating credentials,
     This is primarily done via SASL, though an optional method exists for
     systems which need support for the Thrift login() method.
   - IRoleManager interface has been added which takes over the maintenance
     functions from IAuthenticator. IAuthorizer is mainly unchanged. Auth data
     in systems using the stock internal implementations PasswordAuthenticator
     & CassandraAuthorizer will be automatically converted during upgrade,
     with minimal operator intervention required. Custom implementations will
     require modification, though these can be used in conjunction with the
     stock CassandraRoleManager so providing an IRoleManager implementation
     should not usually be necessary.
   - Fat client support has been removed since we have push notifications to clients
   - cassandra-cli has been removed. Please use cqlsh instead.
   - YamlFileNetworkTopologySnitch has been removed; switch to
     GossipingPropertyFileSnitch instead.
   - CQL2 has been removed entirely in this release (previously deprecated
     in 2.0.0). Please switch to CQL3 if you haven't already done so.
   - The results of CQL3 queries containing an IN restriction will be ordered
     in the normal order and not anymore in the order in which the column values were
     specified in the IN restriction.
   - Some secondary index queries with restrictions on non-indexed clustering
     columns were not requiring ALLOW FILTERING as they should. This has been
     fixed, and those queries now require ALLOW FILTERING (see CASSANDRA-8418
     for details).
   - The SSTableSimpleWriter and SSTableSimpleUnsortedWriter classes have been
     deprecated and will be removed in the next major Cassandra release. You
     should use the CQLSSTableWriter class instead.
   - The sstable2json and json2sstable tools have been deprecated and will be
     removed in the next major Cassandra release. See CASSANDRA-9618
     (https://issues.apache.org/jira/browse/CASSANDRA-9618) for details.
   - nodetool enablehandoff will no longer support a list of data centers starting
     with the next major release. Two new commands will be added, enablehintsfordc and disablehintsfordc,
     to exclude data centers from using hinted handoff when the global status is enabled.
     In cassandra.yaml, hinted_handoff_enabled will no longer support a list of data centers starting
     with the next major release. A new setting will be added, hinted_handoff_disabled_datacenters,
     to exclude data centers when the global status is enabled, see CASSANDRA-9035 for details.


2.1.13
======

New features
------------
    - New options for cqlsh COPY FROM and COPY TO, see CASSANDRA-9303 for details.

2.1.10
=====

New features
------------
   - The syntax TRUNCATE TABLE X is now accepted as an alias for TRUNCATE X


2.1.9
=====

Upgrading
---------
    - cqlsh will now display timestamps with a UTC timezone. Previously,
      timestamps were displayed with the local timezone.
    - Commit log files are no longer recycled by default, due to negative
      performance implications. This can be enabled again with the
      commitlog_segment_recycling option in your cassandra.yaml
    - JMX methods set/getCompactionStrategyClass have been deprecated, use
      set/getCompactionParameters/set/getCompactionParametersJson instead

2.1.8
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.1 if you are upgrading
      from a previous version.


2.1.7
=====



2.1.6
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.1 if you are upgrading
      from a previous version.


2.1.5
=====

Upgrading
---------
    - The option to omit cold sstables with size tiered compaction has been
      removed - it is almost always better to use date tiered compaction for
      workloads that have cold data.

2.1.4
=====

Upgrading
---------
The default JMX config now listens to localhost only. You must enable
the other JMX flags in cassandra-env.sh manually.


2.1.3
=====

Upgrading
---------
    - Prepending a list to a list collection was erroneously resulting in
      the prepended list being reversed upon insertion.  If you were depending
      on this buggy behavior, note that it has been corrected.
    - Incremental replacement of compacted SSTables has been disabled for this
      release.


2.1.2
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.1 if you are upgrading
      from a previous version.


2.1.1
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.1 if you are upgrading
      from a previous version.

New features
------------
   - Netty support for epoll on linux is now enabled.  If for some
     reason you want to disable it pass, the following system property
     -Dcassandra.native.epoll.enabled=false


2.1
===

New features
------------
   - Default data and log locations have changed.  If not set in
     cassandra.yaml, the data file directory, commitlog directory,
     and saved caches directory will default to $CASSANDRA_HOME/data/data,
     $CASSANDRA_HOME/data/commitlog, and $CASSANDRA_HOME/data/saved_caches,
     respectively.  The log directory now defaults to $CASSANDRA_HOME/logs.
     If not set, $CASSANDRA_HOME, defaults to the top-level directory of
     the installation.
     Note that this should only affect source checkouts and tarballs.
     Deb and RPM packages will continue to use /var/lib/cassandra and
     /var/log/cassandra in cassandra.yaml.
   - SSTable data directory name is slightly changed. Each directory will
     have hex string appended after CF name, e.g.
         ks/cf-5be396077b811e3a3ab9dc4b9ac088d/
     This hex string part represents unique ColumnFamily ID.
     Note that existing directories are used as is, so only newly created
     directories after upgrade have new directory name format.
   - Saved key cache files also have ColumnFamily ID in their file name.
   - It is now possible to do incremental repairs, sstables that have been
     repaired are marked with a timestamp and not included in the next
     repair session. Use nodetool repair -par -inc to use this feature.
     A tool to manually mark/unmark sstables as repaired is available in
     tools/bin/sstablerepairedset. This is particularly important when
     using LCS, or any data not repaired in your first incremental repair
     will be put back in L0.
   - Bootstrapping now ensures that range movements are consistent,
     meaning the data for the new node is taken from the node that is no
     longer a responsible for that range of keys.
     If you want the old behavior (due to a lost node perhaps)
     you can set the following property (-Dcassandra.consistent.rangemovement=false)
   - It is now possible to use quoted identifiers in triggers' names.
     WARNING: if you previously used triggers with capital letters in their
     names, then you must quote them from now on.
   - Improved stress tool (http://goo.gl/OTNqiQ)
   - New incremental repair option (http://goo.gl/MjohJp, http://goo.gl/f8jSme)
   - Incremental replacement of compacted SSTables (http://goo.gl/JfDBGW)
   - The row cache can now cache only the head of partitions (http://goo.gl/6TJPH6)
   - Off-heap memtables (http://goo.gl/YT7znJ)
   - CQL improvements and additions: User-defined types, tuple types, 2ndary
     indexing of collections, ... (http://goo.gl/kQl7GW)

Upgrading
---------
   - commitlog_sync_batch_window_in_ms behavior has changed from the
     maximum time to wait between fsync to the minimum time.  We are
     working on making this more user-friendly (see CASSANDRA-9533) but in the
     meantime, this means 2.1 needs a much smaller batch window to keep
     writer threads from starving.  The suggested default is now 2ms.
   - Rolling upgrades from anything pre-2.0.7 is not supported. Furthermore
     pre-2.0 sstables are not supported. This means that before upgrading
     a node on 2.1, this node must be started on 2.0 and
     'nodetool upgdradesstables' must be run (and this even in the case
     of not-rolling upgrades).
   - For size-tiered compaction users, Cassandra now defaults to ignoring
     the coldest 5% of sstables.  This can be customized with the
     cold_reads_to_omit compaction option; 0.0 omits nothing (the old
     behavior) and 1.0 omits everything.
   - Multithreaded compaction has been removed.
   - Counters implementation has been changed, replaced by a safer one with
     less caveats, but different performance characteristics. You might have
     to change your data model to accomodate the new implementation.
     (See https://issues.apache.org/jira/browse/CASSANDRA-6504 and the
     blog post at http://goo.gl/qj8iQl for details).
   - (per-table) index_interval parameter has been replaced with
     min_index_interval and max_index_interval paratemeters. index_interval
     has been deprecated.
   - support for supercolumns has been removed from json2sstable


2.0.11
======

Upgrading
---------
    - Nothing specific to this release, but refer to previous entries if you
      are upgrading from a previous version.

New features
------------
    - DateTieredCompactionStrategy added, optimized for time series data and groups
      data that is written closely in time (CASSANDRA-6602 for details). Consider
      this experimental for now.


2.0.10
======

New features
------------
    - CqlPaginRecordReader and CqlPagingInputFormat have both been removed.
      Use CqlInputFormat instead.
    - If you are using Leveled Compaction, you can now disable doing size-tiered
      compaction in L0 by starting Cassandra with -Dcassandra.disable_stcs_in_l0
      (see CASSANDRA-6621 for details).
    - Shuffle and taketoken have been removed.  For clusters that choose to
      upgrade to vnodes, creating a new datacenter with vnodes and migrating is
      recommended. See http://goo.gl/Sna2S1 for further information.


2.0.9
=====

Upgrading
---------
    - Default values for read_repair_chance and local_read_repair_chance have been
      swapped. Namely, default read_repair_chance is now set to 0.0, and default
      local_read_repair_chance to 0.1.
    - Queries selecting only CQL static columns were (mistakenly) not returning one
      result per row in the partition. This has been fixed and a SELECT DISTINCT
      can be used when only the static column of a partition needs to be fetch
      without fetching the whole partition. But if you use static columns, please
      make sure this won't affect you (see CASSANDRA-7305 for details).


2.0.8
=====

New features
------------
    - New snitches have been used for users of Google Compute Engine and of
      Cloudstack.

Upgrading
---------
    - Nothing specific to this release, but please see 2.0.7 if you are upgrading
      from a previous version.


2.0.7
=====

Upgrading
---------
    - Nothing specific to this release, but please see 2.0.6 if you are upgrading
      from a previous version.


2.0.6
=====

New features
------------
    - CQL now support static columns, allows to batch multiple conditional updates
      and has a new syntax for slicing over multiple clustering columns
      (http://goo.gl/B6qz4j).
    - Repair can be restricted to a set of nodes using the -hosts option in nodetool.
    - A new 'nodetool taketoken' command relocate tokens with vnodes.
    - Hinted handoff can be enabled only for some data-centers (see
      hinted_handoff_enabled in cassandra.yaml)

Upgrading
---------
    - Nothing specific to this release, but please see 2.0.5 if you are upgrading
      from a previous version.


2.0.5
=====

New features
------------
    - Batchlog replay can be, and is throttled by default now.
      See batchlog_replay_throttle_in_kb setting in cassandra.yaml.
    - Scrub can now optionally skip corrupt counter partitions. Please note
      that this will lead to the loss of all the counter updates in the skipped
      partition. See the --skip-corrupted option.

Upgrading
---------
    - If your cluster began on a version before 1.2, check that your secondary
      index SSTables are on version 'ic' before upgrading.  If not, run
      'nodetool upgradesstables' if on 1.2.14 or later, or run 'nodetool
      upgradesstables ks cf' with the keyspace and secondary index named
      explicitly otherwise.  If you don't do this and upgrade to 2.0.x and it
      refuses to start because of 'hf' version files in the secondary index,
      you will need to delete/move them out of the way and recreate the index
      when 2.0.x starts.


2.0.3
=====

New features
------------
    - It's now possible to configure the maximum allowed size of the native
      protocol frames (native_transport_max_frame_size_in_mb in the yaml file).

Upgrading
---------
    - NaN and Infinity are new valid floating point constants in CQL3 and are now reserved
      keywords. In the unlikely case you were using one of them as an identifier (for a
      column, a keyspace or a table), you will now have to double-quote them (see
      http://cassandra.apache.org/doc/cql3/CQL.html#identifiers for "quoted identifiers").
    - The IEndpointStateChangeSubscriber has a new method, beforeChange, that
      any custom implemenations using the class will need to implement.


2.0.2
=====

New features
------------
    - Speculative retry defaults to 99th percentile
      (See blog post at http://www.datastax.com/dev/blog/rapid-read-protection-in-cassandra-2-0-2)
    - Configurable metrics reporting
      (see conf/metrics-reporter-config-sample.yaml)
    - Compaction history and stats are now saved to system keyspace
      (system.compaction_history table). You can access historiy via
      new 'nodetool compactionhistory' command or CQL.

Upgrading
---------
    - Nodetool defaults to Sequential mode for repair operations


2.0.1
=====
Upgrading
---------
    - The default memtable allocation has changed from 1/3 of heap to 1/4
      of heap.  Also, default (single-partition) read and write timeouts
      have been reduced from 10s to 5s and 2s, respectively.


2.0.0
=====

Upgrading
---------
    - Java 7 is now *required*!
    - Upgrading is ONLY supported from Cassandra 1.2.9 or later. This
      goes for sstable compatibility as well as network.  When
      upgrading from an earlier release, upgrade to 1.2.9 first and
      run upgradesstables before proceeding to 2.0.
    - CAS and new features in CQL such as DROP COLUMN assume that cell
      timestamps are microseconds-since-epoch.  Do not use these
      features if you are using client-specified timestamps with some
      other source.
    - Replication and strategy options do not accept unknown options anymore.
      This was already the case for CQL3 in 1.2 but this is now the case for
      thrift too.
    - auto_bootstrap of a single-token node with no initial_token will
      now pick a random token instead of bisecting an existing token
      range.  We recommend upgrading to vnodes; failing that, we
      recommend specifying initial_token.
    - reduce_cache_sizes_at, reduce_cache_capacity_to, and
      flush_largest_memtables_at options have been removed from cassandra.yaml.
    - CacheServiceMBean.reduceCacheSizes() has been removed.
      Use CacheServiceMBean.set{Key,Row}CacheCapacityInMB() instead.
    - authority option in cassandra.yaml has been deprecated since 1.2.0,
      but it has been completely removed in 2.0. Please use 'authorizer' option.
    - ASSUME command has been removed from cqlsh. Use CQL3 blobAsType() and
      typeAsBlob() conversion functions instead.
      See https://cassandra.apache.org/doc/cql3/CQL.html#blobFun for details.
    - Inputting blobs as string constants is now fully deprecated in
      favor of blob constants. Make sure to update your applications to use
      the new syntax while you are still on 1.2 (which supports both string
      and blob constants for blob input) before upgrading to 2.0.
    - index_interval is now moved to ColumnFamily property. You can change value
      with ALTER TABLE ... WITH statement and SSTables written after that will
      have new value. When upgrading, Cassandra will pick up the value defined in
      cassanda.yaml as the default for existing ColumnFamilies, until you explicitly
      set the value for those.
    - The deprecated native_transport_min_threads option has been removed in
      Cassandra.yaml.

Operations
----------
    - VNodes are enabled by default in cassandra.yaml.  initial_token
      for non-vnode deployments has been removed from the example
      yaml, but is still respected if specified.
    - Major compactions, cleanup, scrub, and upgradesstables will interrupt
      any in-progress compactions (but not repair validations) when invoked.
    - Disabling autocompactions by setting min/max compaction threshold to 0
      has been deprecated, instead, use the nodetool commands 'disableautocompaction'
      and 'enableautocompaction' or set the compaction strategy option enabled = false
    - ALTER TABLE DROP has been reenabled for CQL3 tables and has new semantics now.
      See https://cassandra.apache.org/doc/cql3/CQL.html#alterTableStmt and
      https://issues.apache.org/jira/browse/CASSANDRA-3919 for details.
    - CAS uses gc_grace_seconds to determine how long to keep unused paxos
      state around for, or a minimum of three hours.
    - A new hints created metric is tracked per target, replacing countPendingHints
    - After performance testing for CASSANDRA-5727, the default LCS filesize
      has been changed from 5MB to 160MB.
    - cqlsh DESCRIBE SCHEMA no longer outputs the schema of system_* keyspaces;
      use DESCRIBE FULL SCHEMA if you need the schema of system_* keyspaces.
    - CQL2 has been deprecated, and will be removed entirely in 2.2. See
      CASSANDRA-5918 for details.
    - Commit log archiver now assumes the client time stamp to be in microsecond
      precision, during restore. Please refer to commitlog_archiving.properties.


Features
--------
    - Lightweight transactions
      (http://www.datastax.com/dev/blog/lightweight-transactions-in-cassandra-2-0)
    - Alias support has been added to CQL3 SELECT statement. Refer to
      CQL3 documentation (http://cassandra.apache.org/doc/cql3/CQL.html) for details.
    - JEMalloc support (see memory_allocator in cassandra.yaml)
    - Experimental triggers support.  See examples/ for how to use.  "Experimental"
      means "tied closely to internal data structures; we plan to decouple this in
      the future, which will probably break triggers written against this initial
      API."
    - Numerous improvements to CQL3 and a new version of the native protocol. See
      http://www.datastax.com/dev/blog/cql-in-cassandra-2-0 for details.

DDAC 5.1.12

Release notes for DataStax Distribution of Apache Cassandra 5.1.12.

Release notes for DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.12.
Important: DataStax recommends the latest patch release for most environments.

26 December 2018

DDAC 5.1.12 is compatible with Apache Cassandra™ 3.11 and includes production-certified changes to Cassandra.

5.1.12 DDAC database core highlights

  • Skip fetching streamed range in repair during consistent-replace. (DB-2596)
  • Fixed user-defined aggregates (UDAs) that instantiate user-defined types (UDTs) break after restart. (DB-2771)
  • Fixed NullPointerException that can occur during compaction if users use TWCS and allow_unsafe_aggressive_sstable_expiration. (DB-2472)
  • Fixed resource leak related to streaming operations that affects tiered storage users. Excessive number of TieredRowWriter threads causing java.lang.OutOfMemoryError. (DB-2463)
  • General stability improvements:
    • Invalidate chunk cache on SSTable rename. (DB-2594)
    • Fixes to several thread-safety bugs. (DB-2602, DB-2609)
    • Fix for static and regular collision when using same ColumnIdentifier and ComposedTypes. (DB-1630)
  • Upgrade improvements:
    • Fixed handling of deletions for dropped collections in static rows in mixed-version clusters. (DB-2341)
  • Operational improvements:
    • Support for QUORUM/LOCAL_QUORUM consistent replace_address. (DB-1577, DB-2596)
    • Expose information about stored hints by using JMX/nodetool listendpointspendinghints. (DB-1674)
    • New sstablepartitions tool to identify large partitions. (DB-803)
    • Fixed incorrect order of application of nodetool garbagecollect leaves tombstones that should be deleted. (DB-2658)
    • Custom HeapDumpPath is not overwritten. (DB-714)
    • By default, rebuild only locally replicated keyspaces. (DB-2301)

5.1.12 DDAC core changes and enhancements

5.1.12 DDAC resolved issues

  • Custom HeapDumpPath is overwritten. (DB-714)
  • Deleting a static column and adding it back as a non-static column introduces corruption. (DB-1630)
  • Rebuild should not fail when a keyspace is not replicated to other datacenters. (DB-2301)
  • Corrupted static collection deletions for dropped collections in mixed-version clusters. (DB-2341)
  • repair may skip some ranges due to received range cache. (DB-2432)
  • Excessive number of TieredRowWriter threads causing java.lang.OutOfMemoryError (DB-2463)
  • NullPointerException during compaction on table with TimeWindowCompactionStrategy (TWCS). (DB-2472)
  • Prevent potential SSTable corruption with nodetool refresh. (DB-2594)
  • The nodetool gcstats command output incorrectly reports the GC reclaimed metric in bytes, instead of the expected MB. (DB-2598)
  • TypeParser is not thread safe. (DB-2602)
  • STCS, DTCS, TWCS, TMD aren't thread-safe. (DB-2609)
  • Incorrect order of application of nodetool garbagecollect leaves tombstones that should be deleted. (DB-2658)
  • User-defined aggregates (UDAs) that instantiate user-defined types (UDTs) break after restart. (DB-2771)

Enhancements for DDAC 5.1.12

A list of DataStax Distribution of Apache Cassandra 5.1.12 enhancements to Apache Cassandra™ 3.11.

DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.12 includes all changes from previous DDAC releases. These production-certified changes are enhancements to Apache Cassandra™ 3.11. (For updates, see CHANGES.txt.)

  • Legacy sstables with multi block range tombstones create invalid bound sequences (CASSANDRA-14823)
  • Expand range tombstone validation checks to multiple interim request stages (CASSANDRA-14824)
  • Reverse order reads can return incomplete results (CASSANDRA-14803)
  • Avoid calling iter.next() in a loop when notifying indexers about range tombstones (CASSANDRA-14794)
  • Fix purging semi-expired RT boundaries in reversed iterators (CASSANDRA-14672)
  • DESC order reads can fail to return the last Unfiltered in the partition (CASSANDRA-14766)
  • Fix corrupted collection deletions for dropped columns in 3.0 <-> 2.{1,2} messages (CASSANDRA-14568)
  • Handle failures in parallelAllSSTableOperation (cleanup/upgradesstables/etc) (CASSANDRA-14657)
  • Improve TokenMetaData cache populating performance avoid long locking (CASSANDRA-14660)
  • Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
  • sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522)
  • Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
  • Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262)
  • Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)

General upgrade advice for DSE 5.1.12

General upgrade advice for DataStax Enterprise 5.1.12.

Carefully review all planning and upgrade documentation in the Upgrade Guide guide. This general advice applies to the database upgrade and does not replace the upgrade documentation.

PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
------------------------------------------------------------------
QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
SocketServer and ServerSocketReceiver components.

Logback has not been upgraded to avoid breaking deployments and customizations
based on older versions. If you are using vulnerable components you will need
to upgrade to a newer version of Logback or stop using the vulnerable components.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.


DSE 5.1.12
==========

Operations
---------
   - A new property cassandra.range_tombstone_bound_check_chance was added to check for bad range tombstone on a
     percentage of queries.  The default is 0.01 (can be set in range of 0.0 - 1.0 if you wish to adjust).

   - Added "-Ddse.consistent_replace" startup option to support consistent replace by running background repair with
     streaming. Without consistent replace, node replacement only streams one copy of data which may be stale with 
     respect to QUORUM/LOCAL_QUORUM.
     When replacing a down node with "-Ddse.consistent_replace=QUORUM", the replacement node will coordinate repair
     among a quorum of replicas concurrently with replace streaming (or a local quorum of replicas when LOCAL_QUORUM is
     used). Repair will transfer the differences to the replacement node, ensuring it's consistent with other replicas
     when the replacement process is finished, assuming data is inserted with QUORUM/LOCAL_QUORUM.
     Available options: "ONE"(default), "LOCAL_QUORUM", "QUORUM".
     Startup option "-Ddse.consistent_replace.whitelist=<KEYSPACE>,<KEYSPACE.TABLE>,..." can be used to specify
     keyspaces or tables for consistent replace. By default, all keyspaces will be consistent replaced. Specifying
     keyspace without tables means all tables of the keyspace will be consistent replaced.


3.11.4
======

Upgrading
---------
    - The order of static columns in SELECT * has been fixed to match that of 2.0 and 2.1 - they are now sorted
      alphabetically again, by their name, just like regular columns are. If you use prepared statements and
      SELECT * queries, and have both simple and collection static columns in those tables, and are upgrading from an
      earlier 3.0 version, then you might be affected by this change. Please see CASSANDRA-14638 for details.


DSE 5.1.7
==========

Upgrading
---------
    - Automatic fallback of GossipingPropertyFileSnitch to PropertyFileSnitch (cassandra-topology.properties) is
      disabled by default and can be enabled via the -Dcassandra.gpfs.enable_pfs_compatibility_mode=true startup
      flag.

DSE 5.1.6
=========


3.11.3
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 (3.0.X series) or 3.11.1 (3.11.X series) and
      later that have performed range movements (join, decommission, move, etc), should run repair
      on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251
      is correctly propagated to all replicas.
    - Changes to bloom_filter_fp_chance will no longer take effect on existing sstables when the
      node is restarted. Only compactions/upgradesstables regenerates bloom filters and Summaries
      sstable components. See CASSANDRA-11163

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and read_repair_chance
      table options have been deprecated, and will be removed entirely in 4.0. See CASSANDRA-13910
      for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - seed_gossip_probability setting was added to cassandra.yaml.  This setting will pick the percentage of times
      gossip messages are sent to a seed.  This improves the time it takes for gossip changes to
      propagate across the cluster. Defaults to 100% (1.0)
    - Upgrades from DSE 5.0 might have produced unnecessary schema migrations while
      there was at least one DSE 5.0 node in the cluster. It is therefore highly
      recommended to upgrade from DSE 5.0 to at least DSE 5.1.6. The root cause of
      this schema mismatch was a difference in the way how schema digests were computed
      in DSE 5.0 and DSE 5.1. To mitigate this issue, DSE 5.1.6 and newer announce
      DSE 5.0 compatible digests as long as there is at least one DSE 5.0 node in the
      cluster. Once all nodes have been upgraded, the "real" schema version will be
      announced. Note: this fix is only necessary in DSE 5.1 and therefore only applies
      to DSE 5.1. (DB-1477)
    - DSE is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details
    - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
      set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.
    - Improved gossip settling added. On startup DSE waits till all nodes are seen before fully joining the cluster.
      This improves latency spikes when restarting nodes.
    - LeveledCompactionStrategy SSTables will keep their existing level on nodetool refresh, nodetool move,
      and nodetool decommission.

Operations
----------
    - It is now possible to ALTER system_distributed tables
    - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
      The command must be executed on the node where the rebuild operation is running. Streams
      may continue until they finish or timeout.
    - Only MODIFY permission on base is required to update table with MV, internally it reads base
      data and generates updates to MV.

Metrics
-------
   - New storage metrics were added:
     * TotalHintsReplayed: how many hints were successfully replayed on the _target_ node.
     * HintsOnDisk: how many hints are currently persistent on disk on this node. Metric is updated
       for the amount of hints contained in the hints file when hints file is written or removed.
       Values is restored on node startup.

New features
------------
   -  Statistics file component was added to Hint Store in order to provide information about
      amount of hints contained in the hints file withouot replaying it. Stats component is
      completely backwad-compatible; hint files withouot this component will not be counted.
      All new hint files will be created with this component. See DB-853 for more details.


DSE 5.0.10
==========

Upgrading
---------
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on DSE 5.0.10. Upgrading to DSE 5.1 requires at least DSE 5.1.3.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.


3.0.17
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 or later that have performed range movements (join, decommission, move, etc),
      should run repair on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251 is correctly
      propagated to all replicas.

3.0.16
=====

Upgrading
---------
   - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
   - Cassandra is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
     rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
     are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
     options are used. See CASSANDRA-13006 for more details.

3.0.15
=====

Upgrading
---------
   - Nothing specific to this release, but please see previous upgrading sections,
     especially if you are upgrading from 2.2.

Compact Storage
---------------
    - Starting version 4.0, Thrift and COMPACT STORAGE is no longer supported.
      'ALTER ... DROP COMPACT STORAGE' statement makes Compact Tables CQL-compatible,
      exposing internal structure of Thrift/Compact Tables. You can find more details
      on exposed internal structure under:
      http://cassandra.apache.org/doc/latest/cql/appendices.html#appendix-c-dropping-compact-storage

      For uninterrupted cluster upgrades, drivers now support 'NO_COMPACT' startup option.
      Supplying this flag will have same effect as 'DROP COMPACT STORAGE', but only for the
      current connection.

      In order to upgrade, clients supporting a non-compact schema view can be rolled out
      gradually. When all the clients are updated 'ALTER ... DROP COMPACT STORAGE' can be
      executed. After dropping compact storage, ’NO_COMPACT' option will have no effect
      after that.

DSE 5.1.3
=========

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - The table system_auth.resource_role_permissons_index is no longer used and should be dropped
     after all nodes are on 5.1.3. Note that upgrades from DSE 5.0 series since 5.0.10 to DSE
     versions before 5.1.3 are not recommended.
   - Full repairs are now default if no option is specified on nodetool repair, unless
     incremental repair was already run on the table/keyspace being repaired, to maintain
     backward compatibility. Incremental repair may be run on new tables by using the -inc option.
   - Full repairs will no longer run repair unless the --run-anticompaction option is specified
   - Incremental repairs are no longer supported on tables with materialized views or CDC until
     its limitations are addressed. An incremental repair triggered on a base table or
     materialized view run a full repair instead. See CASSANDRA-12888 for details.

Materialized Views (only when upgrading from DSE 5.1.1 or 5.1.2 or any version lower than DSE 5.0.10)
---------------------------------------------------------------------------------------
    - Cassandra will no longer allow dropping columns on tables with Materialized Views.
    - A change was made in the way the Materialized View timestamp is computed, which
      may cause an old deletion to a base column which is view primary key (PK) column
      to not be reflected in the view when repairing the base table post-upgrade. This
      condition is only possible when a column deletion to an MV primary key (PK) column
      not present in the base table PK (via UPDATE base SET view_pk_col = null or DELETE
      view_pk_col FROM base) is missed before the upgrade and received by repair after the upgrade.
      If such column deletions are done on a view PK column which is not a base PK, it's advisable
      to run repair on the base table of all nodes prior to the upgrade. Alternatively it's possible
      to fix potential inconsistencies by running repair on the views after upgrade or drop and
      re-create the views. See CASSANDRA-11500 for more details.
    - Removal of columns not selected in the Materialized View (via UPDATE base SET unselected_column
      = null or DELETE unselected_column FROM base) may not be properly reflected in the view in some
      situations so we advise against doing deletions on base columns not selected in views
      until this is fixed on CASSANDRA-13826.

3.11.2
======

Upgrading
---------
    - Nothing specific to this release, but please see previous upgrading sections.

3.11.1
======

Upgrading
---------
    - Creating Materialized View with filtering on non-primary-key base column
      (added in CASSANDRA-10368) is disabled, because the liveness of view row
      is depending on multiple filtered base non-key columns and base non-key
      column used in view primary-key. This semantic cannot be supported without
      storage format change, see CASSANDRA-13826. For append-only use case, you
      may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"

3.11.0
======

Upgrading
---------
   - Creating Materialized View with filtering on non-primary-key base column
     (added in CASSANDRA-10368) is disabled, because the liveness of view row
     is depending on multiple filtered base non-key columns and base non-key
     column used in view primary-key. This semantic cannot be supported without
     storage format change, see CASSANDRA-13826. For append-only use case, you
     may still use this feature with a startup flag: "-Dcassandra.mv.allow_filtering_nonkey_columns_unsafe=true"
   - ALTER TABLE (ADD/DROP COLUMN) operations concurrent with a read might
     result into data corruption (see CASSANDRA-13004 for more details).
     Fixing this bug required a messaging protocol version bump. By default,
     Cassandra 3.11 will use 3014 version for messaging.

     Since Schema Migrations rely the on exact messaging protocol version
     match between nodes, if you need schema changes during the upgrade
     process, you have to start your nodes with `-Dcassandra.force_3_0_protocol_version=true`
     first, in order to temporarily force a backwards compatible protocol.
     After the whole cluster is upgraded to 3.11, do a rolling
     restart of the cluster without setting that flag.

     3.11 nodes with and withouot the flag set will be able to do schema
     migrations with other 3.x and 3.0.x releases.

     While running the cluster with the flag set to true on 3.11 (in
     compatibility mode), avoid adding or removing any columns to/from
     existing tables.

     If your cluster can do without schema migrations during the upgrade
     time, just start the cluster normally without setting aforementioned
     flag.

     If you are upgrading from 3.0.14+ (of 3.0.x branch), you do not have
     to set an flag while upgrading to ensure schema migrations.
   - The NativeAccessMBean isAvailable method will only return true if the
     native library has been successfully linked. Previously it was returning
     true if JNA could be found but was not taking into account link failures.
   - Primary ranges in the system.size_estimates table are now based on the keyspace
     replication settings and adjacent ranges are no longer merged (CASSANDRA-9639).
   - In 2.1, the default for otc_coalescing_strategy was 'DISABLED'.
     In 2.2 and 3.0, it was changed to 'TIMEHORIZON', but that value was shown
     to be a performance regression. The default for 3.11.0 and newer has
     been reverted to 'DISABLED'. Users upgrading from Cassandra 2.2 or 3.0 should
     be aware that the default has changed.
   - The StorageHook interface has been modified to allow to retrieve read information from
     SSTableReader (CASSANDRA-13120).

3.10
====

New features
------------
   - New `DurationType` (cql duration). See CASSANDRA-11873
   - Runtime modification of concurrent_compactors is now available via nodetool
   - Support for the assignment operators +=/-= has been added for update queries.
   - An Index implementation may now provide a task which runs prior to joining
     the ring. See CASSANDRA-12039
   - Filtering on partition key columns is now also supported for queries without
     secondary indexes.
   - A slow query log has been added: slow queries will be logged at DEBUG level.
     For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms
     in cassandra.yaml.
   - Support for GROUP BY queries has been added.
   - A new compaction-stress tool has been added to test the throughput of compaction
     for any cassandra-stress user schema.  see compaction-stress help for how to use.
   - Compaction can now take into account overlapping tables that don't take part
     in the compaction to look for deleted or overwritten data in the compacted tables.
     Then such data is found, it can be safely discarded, which in turn should enable
     the removal of tombstones over that data.

     The behavior can be engaged in two ways:
       - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies
         single-table compaction on all sstables to discard deleted data in one step.
       - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag,
         which uses overlapping tables as a source of deletions/overwrites during all
         compactions.
     The argument specifies the granularity at which deleted data is to be found:
       - If ROW is specified, only whole deleted rows (or sets of rows) will be
         discarded.
       - If CELL is specified, any columns whose value is overwritten or deleted
         will also be discarded.
       - NONE (default) specifies the old behavior, overlapping tables are not used to
         decide when to discard data.
     Which option to use depends on your workload, both ROW and CELL increase the
     disk load on compaction (especially with the size-tiered compaction strategy),
     with CELL being more resource-intensive. Both should lead to better read
     performance if deleting rows (resp. overwriting or deleting cells) is common.
   - Prepared statements are now persisted in the table prepared_statements in
     the system keyspace. Upon startup, this table is used to preload all
     previously prepared statements - i.e. in many cases clients do not need to
     re-prepare statements against restarted nodes.
   - cqlsh can now connect to older Cassandra versions by downgrading the native
     protocol version. Please note that this is currently not part of our release
     testing and, as a consequence, it is not guaranteed to work in all cases.
     See CASSANDRA-12150 for more details.
   - Snapshots that are automatically taken before a table is dropped or truncated
     will have a "dropped" or "truncated" prefix on their snapshot tag name.
   - Metrics are exposed for successful and failed authentication attempts.
     These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
     and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
     See CASSANDRA-11424 for details
   - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0.
   - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details.

Upgrading
---------
    - Support for alter types of already defined tables and of UDTs fields has been disabled.
      If it is necessary to return a different type, please use casting instead. See
      CASSANDRA-12443 for more details.
    - Specifying the default_time_to_live option when creating or altering a
      materialized view was erroneously accepted (and ignored). It is now
      properly rejected.
    - Only Java and JavaScript are now supported UDF languages.
      The sandbox in 3.0 already prevented the use of script languages except Java
      and JavaScript.
    - Compaction now correctly drops sstables out of CompactionTask when there
      isn't enough disk space to perform the full compaction.  This should reduce
      pending compaction tasks on systems with little remaining disk space.
    - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the
      "full" request time on the coordinator.  Previously, they only covered the time from
      when the coordinator sent a message to a replica until the time that the replica
      responded.  Additionally, the previous behavior was to reset the timeout when performing
      a read repair, making a second read to fix a short read, and when subranges were read
      as part of a range scan or secondary index query.  In 3.10 and higher, the timeout
      is no longer reset for these "subqueries".  The entire request must complete within
      the specified timeout.  As a consequence, your timeouts may need to be adjusted
      to account for this.  See CASSANDRA-12256 for more details.
    - Logs written to stdout are now consistent with logs written to files.
      Time is now local (it was UTC on the console and local in files). Date, thread, file
      and line info where added to stdout. (see CASSANDRA-12004)
    - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided.
      The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on
      that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided
      on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable
      accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues).
    - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like
      commit log replay. This may break existing 3rd party tools and clients. In order to startup
      a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or
      DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner,
      snitch, encryption context. Client initialization just applies the configuration but does not
      setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are
      deprecated now, use one of the appropiate new methods in DatabaseDescriptor.
    - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from
      timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms
      property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property.
    - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format
      (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873).

3.8
===

New features
------------
   - Shared pool threads are now named according to the stage they are executing
     tasks for. Thread names mentioned in traced queries change accordingly.
   - A new option has been added to cassandra-stress "-rate fixed={number}/s"
     that forces a scheduled rate of operations/sec over time. Using this, stress can
     accurately account for coordinated ommission from the stress process.
   - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle="
   - hdr histograms have been added to stress runs, it's output can be saved to disk using:
     "-log hdrfile=" option. This histogram includes response/service/wait times when used with the
     fixed or throttle rate options.  The histogram file can be plotted on
     http://hdrhistogram.github.io/HdrHistogram/plotFiles.html
   - TimeWindowCompactionStrategy has been added. This has proven to be a better approach
     to time series compaction and new tables should use this instead of DTCS. See
     CASSANDRA-9666 for details.
   - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and
     a brief explanation of on-disk locations for archived data in CommitLog form. This can
     be enabled via ALTER TABLE ... WITH cdc=true.
     Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to
     the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables
     will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached
     between unflushed CommitLogSegments and cdc_raw.
     NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version
     cluster as it will lead to exceptions which can interrupt traffic. Once all nodes
     have been upgraded to 3.8 it is safe to enable this feature and restart the cluster.

Upgrading
---------
   - The ReversedType behaviour has been corrected for clustering columns of
     BYTES type containing empty value. Scrub should be run on the existing
     SSTables containing a descending clustering column of BYTES type to correct
     their ordering. See CASSANDRA-12127 for more details.
   - Ec2MultiRegionSnitch will no longer automatically set broadcast_rpc_address
     to the public instance IP if this property is defined on cassandra.yaml.
   - The name "json" and "distinct" are not valid anymore a user-defined function
     names (they are still valid as column name however). In the unlikely case where
     you had defined functions with such names, you will need to recreate
     those under a different name, change your code to use the new names and
     drop the old versions, and this _before_ upgrade (see CASSANDRA-10783 for more
     details).

Deprecation
-----------
   - DateTieredCompactionStrategy has been deprecated - new tables should use
     TimeWindowCompactionStrategy. Note that migrating an existing DTCS-table to TWCS might
     cause increased compaction load for a while after the migration so make sure you run
     tests before migrating. Read CASSANDRA-9666 for background on this.

3.7
===

Upgrading
---------
   - A maximum size for SSTables values has been introduced, to prevent out of memory
     exceptions when reading corrupt SSTables. This maximum size can be set via
     max_value_size_in_mb in cassandra.yaml. The default is 256MB, which matches the default
     value of native_transport_max_frame_size_in_mb. SSTables will be considered corrupt if
     they contain values whose size exceeds this limit. See CASSANDRA-9530 for more details.


3.6
=====

New features
------------
   - JMX connections can now use the same auth mechanisms as CQL clients. New options
     in cassandra-env.(sh|ps1) enable JMX authentication and authorization to be delegated
     to the IAuthenticator and IAuthorizer configured in cassandra.yaml. The default settings
     still only expose JMX locally, and use the JVM's own security mechanisms when remote
     connections are permitted. For more details on how to enable the new options, see the
     comments in cassandra-env.sh. A new class of IResource, JMXResource, is provided for
     the purposes of GRANT/REVOKE via CQL. See CASSANDRA-10091 for more details.
     Also, directly setting JMX remote port via the com.sun.management.jmxremote.port system
     property at startup is deprecated. See CASSANDRA-11725 for more details.
   - JSON timestamps are now in UTC and contain the timezone information, see CASSANDRA-11137 for more details.
   - Collision checks are performed when joining the token ring, regardless of whether
     the node should bootstrap. Additionally, replace_address can legitimately be used
     without bootstrapping to help with recovery of nodes with partially failed disks.
     See CASSANDRA-10134 for more details.
   - Key cache will only hold indexed entries up to the size configured by
     column_index_cache_size_in_kb in cassandra.yaml in memory. Larger indexed entries
     will never go into memory. See CASSANDRA-11206 for more details.
   - For tables having a default_time_to_live specifying a TTL of 0 will remove the TTL
     from the inserted or updated values.
   - Startup is now aborted if corrupted transaction log files are found. The details
     of the affected log files are now logged, allowing the operator to decide how
     to resolve the situation.
   - Filtering expressions are made more pluggable and can be added programatically via
     a QueryHandler implementation. See CASSANDRA-11295 for more details.



3.4
===

New features
------------
    - Internal authentication now supports caching of encrypted credentials.
      Reference cassandra.yaml:credentials_validity_in_ms
    - Remote configuration of auth caches via JMX can be disabled using the
      the system property cassandra.disable_auth_caches_remote_configuration
    - sstabledump tool is added to be 3.0 version of former sstable2json. The tool only
      supports v3.0+ SSTables. See tool's help for more detail.

Upgrading
---------
    - Nothing specific to 3.4 but please see previous versions upgrading section,
      especially if you are upgrading from 2.2.

Deprecation
-----------
    - The mbean interfaces org.apache.cassandra.auth.PermissionsCacheMBean and
      org.apache.cassandra.auth.RolesCacheMBean are deprecated in favor of
      org.apache.cassandra.auth.AuthCacheMBean. This generalized interface is
      common across all caches in the auth subsystem. The specific mbean interfaces
      for each individual cache will be removed in a subsequent major version.


3.2
===

New features
------------
   - We now make sure that a token does not exist in several data directories. This
     means that we run one compaction strategy per data_file_directory and we use
     one thread per directory to flush. Use nodetool relocatesstables to make sure your
     tokens are in the correct place, or just wait and compaction will handle it. See
     CASSANDRA-6696 for more details.
   - bound maximum in-flight commit log replay mutation bytes to 64 megabytes
     tunable via cassandra.commitlog_max_outstanding_replay_bytes
   - Support for type casting has been added to the selection clause.
   - Hinted handoff now supports compression. Reference cassandra.yaml:hints_compression.
     Note: hints compression is currently disabled by default.

Upgrading
---------
   - The compression ratio metrics computation has been modified to be more accurate.
   - Running Cassandra as root is prevented by default.
   - JVM options are moved from cassandra-env.(sh|ps1) to jvm.options file

Deprecation
-----------
   - The Thrift API is deprecated and will be removed in Cassandra 4.0.


3.1
=====

Upgrading
---------
   - The return value of SelectStatement::getLimit as been changed from DataLimits
     to int.
   - Custom index implementation should be aware that the method Indexer::indexes()
     has been removed as its contract was misleading and all custom implementation
     should have almost surely returned true inconditionally for that method.
   - GC logging is now enabled by default (you can disable it in the jvm.options
     file if you prefer).


3.0
===

New features
------------
   - EACH_QUORUM is now a supported consistency level for read requests.
   - Support for IN restrictions on any partition key component or clustering key
     as well as support for EQ and IN multicolumn restrictions has been added to
     UPDATE and DELETE statement.
   - Support for single-column and multi-colum slice restrictions (>, >=, <= and <)
     has been added to DELETE statements
   - nodetool rebuild_index accepts the index argument without
     the redundant table name
   - Materialized Views, which allow for server-side denormalization, is now
     available. Materialized views provide an alternative to secondary indexes
     for non-primary key queries, and perform much better for indexing high
     cardinality columns.
     See http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views
   - Hinted handoff has been completely rewritten. Hints are now stored in flat
     files, with less overhead for storage and more efficient dispatch.
     See CASSANDRA-6230 for full details.
   - Option to not purge unrepaired tombstones. To avoid users having data resurrected
     if repair has not been run within gc_grace_seconds, an option has been added to
     only allow tombstones from repaired sstables to be purged. To enable, set the
     compaction option 'only_purge_repaired_tombstones':true but keep in mind that if
     you do not run repair for a long time, you will keep all tombstones around which
     can cause other problems.
   - Enabled warning on GC taking longer than 1000ms. See
     cassandra.yaml:gc_warn_threshold_in_ms

Upgrading
---------
   - Clients must use the native protocol version 3 when upgrading from 2.2.X as
     the native protocol version 4 is not compatible between 2.2.X and 3.Y. See
     https://www.mail-archive.com/user@cassandra.apache.org/msg45381.html for details.
   - A new argument of type InetAdress has been added to IAuthenticator::newSaslNegotiator,
     representing the IP address of the client attempting authentication. It will be a breaking
     change for any custom implementations.
   - token-generator tool has been removed.
   - Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal to 2.1.9,
     or Cassandra 2.2 versions greater or equal to 2.2.2. Upgrade from Cassandra 2.0 and
     older versions is not supported.
   - The 'memtable_allocation_type: offheap_objects' option has been removed. It should
     be re-introduced in a future release and you can follow CASSANDRA-9472 to know more.
   - Configuration parameter memory_allocator in cassandra.yaml has been removed.
   - The native protocol versions 1 and 2 are not supported anymore.
   - Max mutation size is now configurable via max_mutation_size_in_kb setting in
     cassandra.yaml; the default is half the size commitlog_segment_size_in_mb * 1024.
   - 3.0 requires Java 8u40 or later.
   - Garbage collection options were moved from cassandra-env to jvm.options file.
   - New transaction log files have been introduced to replace the compactions_in_progress
     system table, temporary file markers (tmp and tmplink) and sstable ancerstors.
     Therefore, compaction metadata no longer contains ancestors. Transaction log files
     list sstable descriptors involved in compactions and other operations such as flushing
     and streaming. Use the sstableutil tool to list any sstable files currently involved
     in operations not yet completed, which previously would have been marked as temporary.
     A transaction log file contains one sstable per line, with the prefix "add:" or "remove:".
     They also contain a special line "commit", only inserted at the end when the transaction
     is committed. On startup we use these files to cleanup any partial transactions that were
     in progress when the process exited. If the commit line is found, we keep new sstables
     (those with the "add" prefix) and delete the old sstables (those with the "remove" prefix),
     vice-versa if the commit line is missing. Should you lose or delete these log files,
     both old and new sstable files will be kept as live files, which will result in duplicated
     sstables. These files are protected by incremental checksums so you should not manually
     edit them. When restoring a full backup or moving sstable files, you should clean-up
     any left over transactions and their temporary files first. You can use this command:
      ===> sstableutil -c ks table
     See CASSANDRA-7066 for full details.
   - New write stages have been added for batchlog and materialized view mutations
     you can set their size in cassandra.yaml
   - User defined functions are now executed in a sandbox.
     To use UDFs and UDAs, you have to enable them in cassandra.yaml.
   - New SSTable version 'la' with improved bloom-filter false-positive handling
     compared to previous version 'ka' used in 2.2 and 2.1. Running sstableupgrade
     is not necessary but recommended.
   - Before upgrading to 3.0, make sure that your cluster is in complete agreement
     (schema versions outputted by `nodetool describecluster` are all the same).
   - Schema metadata is now stored in the new `system_schema` keyspace, and
     legacy `system.schema_*` tables are now gone; see CASSANDRA-6717 for details.
   - Pig's support has been removed.
   - Hadoop BulkOutputFormat and BulkRecordWriter have been removed; use
     CqlBulkOutputFormat and CqlBulkRecordWriter instead.
   - Hadoop ColumnFamilyInputFormat and ColumnFamilyOutputFormat have been removed;
     use CqlInputFormat and CqlOutputFormat instead.
   - Hadoop ColumnFamilyRecordReader and ColumnFamilyRecordWriter have been removed;
     use CqlRecordReader and CqlRecordWriter instead.
   - hinted_handoff_enabled in cassandra.yaml no longer supports a list of data centers.
     To specify a list of excluded data centers when hinted_handoff_enabled is set to true,
     use hinted_handoff_disabled_datacenters, see CASSANDRA-9035 for details.
   - The `sstable_compression` and `chunk_length_kb` compression options have been deprecated.
     The new options are `class` and `chunk_length_in_kb`. Disabling compression should now
     be done by setting the new option `enabled` to `false`.
   - The compression option `crc_check_chance` became a top-level table option, but is currently
     enforced only against tables with enabled compression.
   - Only map syntax is now allowed for caching options. ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax
     has been deprecated since 2.1.0 and is being removed in 3.0.0.
   - The 'index_interval' option for 'CREATE TABLE' statements, which has been deprecated
     since 2.1 and replaced with the 'min_index_interval' and 'max_index_interval' options,
     has now been removed.
   - Batchlog entries are now stored in a new table - system.batches.
     The old one has been deprecated.
   - JMX methods set/getCompactionStrategyClass have been removed, use
     set/getCompactionParameters or set/getCompactionParametersJson instead.
   - SizeTieredCompactionStrategy parameter cold_reads_to_omit has been removed.
   - The secondary index API has been comprehensively reworked. This will be a breaking
     change for any custom index implementations, which should now look to implement
     the new org.apache.cassandra.index.Index interface. New syntax has been added to create
     and query row-based indexes, which are not explicitly linked to a single column in the
     base table.

DDAC 5.1.11

Release notes for DataStax Distribution of Apache Cassandra 5.1.11.

Release notes for DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.11.
Important: DataStax recommends the latest patch release for most environments.

14 September 2018

DDAC is compatible with Apache Cassandra™ 3.11.0 and adds production-certified changes and enhancements.

5.1.11 DDAC core changes and enhancements

  • Connections on non-serialization errors are not dropped. (DB-2233)
  • Create a log message when DDL statements are executed. (DB-2383)

5.1.11 DDAC resolved issues

  • Set MX4J_ADDRESS to 127.0.0.1 if not explicitly set. (DB-1950)
  • Digest mismatch for same data between nodes with flushed memtables and nodes with non-flushed memtables. (DB-1980)
  • Fix handling of start bound in legacy paged queries. (DB-1984)
  • Move TWCS message "No compaction necessary for bucket size" to Trace level or NoSpam. (DB-2022)
  • Limit max cached direct buffer on NIO to 1 MB. (DB-2028)
  • Compaction strategy instantiation errors don't generate meaningful error messages, instead return only InvocationTargetException. (DB-2404)
  • Non-portable syntax (MX4J bash-isms) in cassandra-env.sh broke service scripts. (DB-2123)
  • nodetool upgradesstables fails with 20-year TTL. After upgrade to 5.1.11, take required action. (DB-2109)
  • Add missing equality sign to SASI schema snapshot. (DB-2129)
  • For tables using DSE Tiered Storage, nodetool cleanup places cleaned SSTables in the wrong tier. (DB-2173)
  • sstableloader options assume the RPC/native (client) interface is the same as the internode (node-to-node) interface. (DB-2184)
  • Audit events for CREATE ROLE and ALTER ROLE with incorrect spacing exposes PASSWORD in plain text. (DB-2285)
  • Client warnings are not always propagated via LocalSessionWrapper. (DB-2304)
  • Timestamps inserted with ISO 8601 format are saved with wrong millisecond value. (DB-2312)
  • Compaction fails with IllegalArgumentException: null. (DB-2329)
  • BulkLoader class exits without printing the stack trace for throwable error. (DB-2377)
  • Select Hadoop libraries

    Built-in Hadoop and Bring-Your-Own-Hadoop (BYOH) were deprecated in DataStax Enterprise (DSE) 5.0, and were removed in DSE 5.1. Hadoop removal from DSE 5.1 and later means that DSE does not allow for the startup of Hadoop services previously included in DSE, including MapReduce JobTracker and TaskTracker.

    However, DSE has supported built-in Spark since DSE 4.5 and Bring-Your-Own-Spark (BYOS) since DSE 5.0, and that support continues today. Because Spark depends on certain Hadoop libraries on the server and the client, DSE continues to ship with Hadoop libraries that are required for running Spark and BYOS.

    To view the included Hadoop libraries, see DataStax Enterprise 5.1.x third-party software.

Enhancements for DDAC 5.1.11

A list of DataStax Distribution of Apache Cassandra 5.1.11 enhancements to Apache Cassandra™ 3.1.10.

DataStax Distribution of Apache Cassandra™ (DDAC) 5.1.11 includes all changes from previous releases. These production-certified changes are enhancements to Apache Cassandra™ 3.11.0. (For updates, see CHANGES.txt.)

  • Fix static column order for SELECT * wildcard queries (CASSANDRA-14638)
  • sstableloader should use discovered broadcast address to connect intra-cluster (CASSANDRA-14522)
  • Fix reading columns with non-UTF names from schema (CASSANDRA-14468)
  • Validate supported column type with SASI analyzer (CASSANDRA-13669)
  • Remove BTree.Builder Recycler to reduce memory usage (CASSANDRA-13929)
  • Reduce nodetool GC thread count (CASSANDRA-14475)
  • Fix New SASI view creation during Index Redistribution (CASSANDRA-14055)
  • Remove string formatting lines from BufferPool hot path (CASSANDRA-14416)
  • Update metrics to 3.1.5 (CASSANDRA-12924)
  • Detect OpenJDK jvm type and architecture (CASSANDRA-12793)
  • Don't use guava collections in the non-system keyspace jmx attributes (CASSANDRA-12271)
  • Fix corrupted static collection deletions in 3.0 -> 2.{1,2} messages (CASSANDRA-14568)
  • Fix potential IndexOutOfBoundsException with counters (CASSANDRA-14167)
  • Always close RT markers returned by ReadCommand#executeLocally() (CASSANDRA-14515)
  • Reverse order queries with range tombstones can cause data loss (CASSANDRA-14513)
  • Fix regression of lagging commitlog flush log message (CASSANDRA-14451)
  • Add Missing dependencies in pom-all (CASSANDRA-14422)
  • Cleanup StartupClusterConnectivityChecker and PING Verb (CASSANDRA-14447)
  • Fix deprecated repair error notifications from 3.x clusters to legacy JMX clients (CASSANDRA-13121)
  • Cassandra not starting when using enhanced startup scripts in windows (CASSANDRA-14418)
  • Fix progress stats and units in compactionstats (CASSANDRA-12244)
  • Better handle missing partition columns in system_schema.columns (CASSANDRA-14379)
  • Delay hints store excise by write timeout to avoid race with decommission (CASSANDRA-13740)
  • Incorrect counting of pending messages in OutboundTcpConnection (CASSANDRA-11551)
  • Fix compaction failure caused by reading un-flushed data (CASSANDRA-12743)

General upgrade advice for DSE 5.1.11

General upgrade advice for DataStax Enterprise 5.1.11.

Carefully review all planning and upgrade documentation in the Upgrade Guide guide. This general advice applies to the database upgrade and does not replace the upgrade documentation.

PLEASE READ: MAXIMUM TTL EXPIRATION DATE NOTICE (CASSANDRA-14092)
------------------------------------------------------------------
(General upgrading instructions are available in the next section)

The maximum expiration timestamp that can be represented by the storage engine is
2038-01-19T03:14:06+00:00, which means that inserts with TTL thatl expire after
this date are not currently supported. By default, INSERTS with TTL exceeding the
maximum supported date are rejected, but it's possible to choose a different
 expiration overflow policy. See CASSANDRA-14092.txt for more details.

Prior to 5.0.12 (5.0.X) and 5.1.7 (5.1.x) there was no protection against INSERTS
with TTL expiring after the maximum supported date, causing the expiration time
field to overflow and the records to expire immediately. Clusters in the 4.X and
lower series are not subject to this when assertions are enabled. Backed up SSTables
can be potentially recovered and recovery instructions can be found on the
CASSANDRA-14092.txt file.

If you use or plan to use very large TTLS (10 to 20 years), read CASSANDRA-14092.txt
for more information.

PLEASE READ: CVE-2017-5929 LOGBACK BEFORE 1.2.0 SERIALIZATION VULNERABILITY
------------------------------------------------------------------
QOS.ch Logback before 1.2.0 has a serialization vulnerability affecting the
SocketServer and ServerSocketReceiver components.

Logback has not been upgraded to avoid breaking deployments and customizations
based on older versions. If you are using vulnerable components you will need
to upgrade to a newer version of Logback or stop using the vulnerable components.

GENERAL UPGRADING ADVICE FOR ANY VERSION
========================================

Snapshotting is fast (especially if you have JNA installed) and takes
effectively zero disk space until you start compacting the live data
files again.  Thus, best practice is to ALWAYS snapshot before any
upgrade, just in case you need to roll back to the previous version.
(Cassandra version X + 1 will always be able to read data files created
by version X, but the inverse is not necessarily the case.)

When upgrading major versions of Cassandra, you will be unable to
restore snapshots created with the previous major version using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.

3.11.4
======

Upgrading
---------
    - The order of static columns in SELECT * has been fixed to match that of 2.0 and 2.1 - they are now sorted
      alphabetically again, by their name, just like regular columns are. If you use prepared statements and
      SELECT * queries, and have both simple and collection static columns in those tables, and are upgrading from an
      earlier 3.0 version, then you might be affected by this change. Please see CASSANDRA-14638 for details.


DSE 5.1.7
==========

Upgrading
---------
    - Automatic fallback of GossipingPropertyFileSnitch to PropertyFileSnitch (cassandra-topology.properties) is
      disabled by default and can be enabled via the -Dcassandra.gpfs.enable_pfs_compatibility_mode=true startup
      flag.


DSE 5.1.6
=========


3.11.3
=====

Upgrading
---------
    - Materialized view users upgrading from 3.0.15 (3.0.X series) or 3.11.1 (3.11.X series) and
      later that have performed range movements (join, decommission, move, etc), should run repair
      on the base tables, and subsequently on the views to ensure data affected by CASSANDRA-14251
      is correctly propagated to all replicas.
    - Changes to bloom_filter_fp_chance will no longer take effect on existing sstables when the
      node is restarted. Only compactions/upgradesstables regenerates bloom filters and Summaries
      sstable components. See CASSANDRA-11163

Deprecation
-----------
    - Background read repair has been deprecated. dclocal_read_repair_chance and read_repair_chance
      table options have been deprecated, and will be removed entirely in 4.0. See CASSANDRA-13910
      for details.

3.11.2
======

Upgrading
---------
    - See MAXIMUM TTL EXPIRATION DATE NOTICE above.
    - seed_gossip_probability setting was added to cassandra.yaml.  This setting will pick the percentage of times
      gossip messages are sent to a seed.  This improves the time it takes for gossip changes to
      propagate across the cluster. Defaults to 100% (1.0)
    - Upgrades from DSE 5.0 might have produced unnecessary schema migrations while
      there was at least one DSE 5.0 node in the cluster. It is therefore highly
      recommended to upgrade from DSE 5.0 to at least DSE 5.1.6. The root cause of
      this schema mismatch was a difference in the way how schema digests were computed
      in DSE 5.0 and DSE 5.1. To mitigate this issue, DSE 5.1.6 and newer announce
      DSE 5.0 compatible digests as long as there is at least one DSE 5.0 node in the
      cluster. Once all nodes have been upgraded, the "real" schema version will be
      announced. Note: this fix is only necessary in DSE 5.1 and therefore only applies
      to DSE 5.1. (DB-1477)
    - DSE is now relying on the JVM options to properly shutdown on OutOfMemoryError. By default it will
      rely on the OnOutOfMemoryError option as the ExitOnOutOfMemoryError and CrashOnOutOfMemoryError options
      are not supported by the older 1.7 and 1.8 JVMs. A warning will be logged at startup if none of those JVM
      options are used. See CASSANDRA-13006 for more details
    - DSE is logging by default a heap histogram on OutOfMemoryError. To disable that behavior
      set the 'cassandra.printHeapHistogramOnOutOfMemoryError' System property to 'false'.
    - Improved gossip settling added. On startup DSE waits till all nodes are seen before fully joining the cluster.
      This improves latency spikes when restarting nodes.
    - LeveledCompactionStrategy SSTables will keep their existing level on nodetool refresh, nodetool move,
      and nodetool decommission.

Operations
----------
    - It is now possible to ALTER system_distributed tables
    - New command 'nodetool abortrebuild' allows to abort a currently running rebuild operation.
      The command must be executed on the node where the rebuild operation is running. Streams
      may continue until they finish or timeout.
    - Only MODIFY permission on base is required to update table with MV, internally it reads base
      data and generates updates to MV.

Metrics
-------
   - New storage metrics were added:
     * TotalHintsReplayed: how many hints were successfully replayed on the _target_ node.
     * HintsOnDisk: how many hints are currently persistent on disk on this node. Metric is updated
       for the amount of hints contained in the hints file when hints file is written or removed.
       Values is restored on node startup.

New features
------------
   -  Statistics file component was added to Hint Store in order to provide information about
      amount of hints contained in the hints file withouot replaying it. Stats component is
      completely backwad-compatible; hint files withouot this component will not be counted.
      All new hint files will be created with this component. See DB-853 for more details.