Release notes

DataStax Enterprise release notes cover components, changes and enhancements, issues, and resolved issues for DataStax Enterprise 4.5x.

Before you upgrade

The latest version of DataStax Enterprise 4.5 is 4.5.9.

DataStax Enterprise 4.5.9 release notes 

Components

  • Cassandra: 2.0.16.762
  • Scala: 2.10.5
  • Shark: 0.9.1.5
  • Solr: 4.6.0.2.168
  • Tomcat: 6.0.44
Changes and enhancements
  • Verify All Unique Key Elements are Indexed in Solr Schema (DSP-6460)
  • Use BIO Tomcat connector to improve DSE Search throughput (DSP-6302)

Resolved issues

  • _docBoost prevents docs from being indexed (DSP-6486)
  • Range tombstones may cause unwanted deletes from the index (DSP-6301)
  • load_max_time_per_core is too low by default (DSP-6260)
  • Bootstrapping TDE nodes try to read from themselves (DSP-5701)
  • Bootstrap of Cassandra only nodes delayed by unnecessary Solr index build (DSP-5273)
  • Audit log recorded table_name as null when deleting index (DSP-4721)
  • Index updates fail due to Assertion in Cassandra (DSP-4663)
  • Bad token to string in log message (DSP-2970)
Patches
  • Backport CASSANDRA-8013 to 2.0 (CASSANDRA-10144)
  • Make getFullyExpiredSSTables less expensive (CASSANDRA-9882)
  • Add tool to find why expired sstables are not getting dropped (CASSANDRA-10015)
  • Remove erroneous pending HH tasks from tpstats/jmx (CASSANDRA-9129)
  • Don't cast expected bf size to an int (CASSANDRA-9959)
  • Log when messages are dropped due to cross_node_timeout (CASSANDRA-9793)
  • checkForEndpointCollision fails for legitimate collisions (CASSANDRA-9765)
  • Complete CASSANDRA-8448 fix (CASSANDRA-9519)
  • Don't include auth credentials in debug log (CASSANDRA-9682)
  • Can't transition from write survey to normal mode (CASSANDRA-9740)
  • Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
  • Add listen_address to system.local (CASSANDRA-9603)
  • Bug fixes to resultset metadata construction (CASSANDRA-9636)
  • Fix setting 'durable_writes' in ALTER KEYSPACE (CASSANDRA-9560)
  • Avoid ballot clash in Paxos (CASSANDRA-9649)
  • Improve trace messages for RR (CASSANDRA-9479)
  • Fix suboptimal secondary index selection when restricted clustering column is also indexed (CASSANDRA-9631)
  • (cqlsh) Add min_threshold to DTCS option autocomplete (CASSANDRA-9385)
  • Fix error message when attempting to create an index on a column in a COMPACT STORAGE table with clustering columns (CASSANDRA-9527)
  • 'WITH WITH' in alter keyspace statements causes NPE (CASSANDRA-9565)
  • Display min timestamp in sstablemetadata viewer (CASSANDRA-6767)
  • Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
  • Fix growing pending background compaction (CASSANDRA-9662)
  • Don't track hotness when opening from snapshot for validation (CASSANDRA-9382)
  • Expose some internals of SelectStatement for inspection (CASSANDRA-9532)
  • ArrivalWindow should use primitives (CASSANDRA-9496)
  • Periodically submit background compaction tasks (CASSANDRA-9592)
  • Set HAS_MORE_PAGES flag to false when PagingState is null (CASSANDRA-9571)
  • Backport indexed value validation fix from CASSANDRA-9057 (CASSANDRA-9564)
  • Don't accumulate more range than necessary in RangeTombstone.Tracker (CASSANDRA-9486)
  • Add broadcast and rpc addresses to system.local (CASSANDRA-9436)
  • Always mark sstable suspect when corrupted (CASSANDRA-9478)
  • Add database users and permissions to CQL3 documentation (CASSANDRA-7558)
  • Allow JVM_OPTS to be passed to standalone tools (CASSANDRA-5969)
  • Fix bad condition in RangeTombstoneList (CASSANDRA-9485)
  • Fix potential StackOverflow when setting CrcCheckChance over JMX (CASSANDRA-9488)
  • Fix null static columns in pages after the first, paged reversed queries (CASSANDRA-8502)
  • Fix failing bound statement after adding a collection (CASSANDRA-9411)
  • Fix counting cache serialization in request metrics (CASSANDRA-9466)
  • (cqlsh) Add LOGIN command to switch users (CASSANDRA-7212)
  • Clone SliceQueryFilter in AbstractReadCommand implementations (CASSANDRA-8940)
  • Push correct protocol notification for DROP INDEX (CASSANDRA-9310)
  • token-generator - generated tokens too long (CASSANDRA-9300)
  • Add option not to validate atoms during scrub (CASSANDRA-9406)
  • Backport CASSANDRA-3569 (CASSANDRA-9455)
  • Fix counting of tombstones for TombstoneOverwhelmingException (CASSANDRA-9299)
  • Fix ReconnectableSnitch reconnecting to peers during upgrade (CASSANDRA-6702)
  • Include keyspace and table name in error log for collections over the size limit (CASSANDRA-9286)
  • Avoid potential overlap in LCS with single-partition sstables (CASSANDRA-9322)
  • Log warning message when a table is queried before the schema has fully propagated (CASSANDRA-9136)
  • Overload SecondaryIndex#indexes to accept the column definition (CASSANDRA-9314)
  • (cqlsh) Add SERIAL and LOCAL_SERIAL consistency levels (CASSANDRA-8051)
  • Fix index selection during rebuild with certain table layouts (CASSANDRA-9281)
  • Fix partition-level-delete-only workload accounting (CASSANDRA-9194)
  • Allow scrub to handle corrupted compressed chunks (CASSANDRA-9140)
  • Fix assertion error when resetlocalschema is run during repair (CASSANDRA-9249)
  • Disable single sstable tombstone compactions for DTCS by default (CASSANDRA-9234)
  • Do more agressive ttl expiration checks to be able to drop more sstables (CASSANDRA-8243)
  • IncomingTcpConnection thread is not named (CASSANDRA-9262)
  • Close incoming connections when MessagingService is stopped (CASSANDRA-9238)
  • Avoid overflow when calculating max sstable size in LCS (CASSANDRA-9235)
  • Make sstable blacklisting work with compression (CASSANDRA-9138)
  • Do not attempt to rebuild indexes if no index accepts any column (CASSANDRA-9196)
  • Don't initiate snitch reconnection for dead states (CASSANDRA-7292)
  • Fix ArrayIndexOutOfBoundsException in CQLSSTableWriter (CASSANDRA-8978)
  • Add shutdown gossip state to prevent timeouts during rolling restarts (CASSANDRA-8336)
  • Fix running with java.net.preferIPv6Addresses=true (CASSANDRA-9137)
  • Fix failed bootstrap/replace attempts being persisted in system.peers (CASSANDRA-9180)
  • Flush system.IndexInfo after marking index built (CASSANDRA-9128)
  • Fix updates to min/max_compaction_threshold through cassandra-cli (CASSANDRA-8102)
  • Backport CASSANDRA-6863 without digest updates (CASSANDRA-8989)
  • Don't include tmp files when doing offline relevel (CASSANDRA-9088)
  • Use the proper CAS WriteType when finishing a previous round during Paxos preparation (CASSANDRA-8672)
  • Avoid race in cancelling compactions (CASSANDRA-9070)
  • More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  • Don't set clientMode to true when bulk-loading sstables to avoid against invalid keyspaces or tables (CASSANDRA-8950)
  • Allow specifying the tmp dir (CASSANDRA-7712)
  • Improve compaction estimated tasks estimation (CASSANDRA-8904)
  • Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  • Expose commit log archive status via JMX (CASSANDRA-8734)
  • Provide better exceptions for invalid replication strategy parameters (CASSANDRA-8909) (CASSANDRA-8808)
  • Fix MT mismatch between empty and GC-able data (CASSANDRA-8979)
  • Fix incorrect validation when snapshotting single table (CASSANDRA-8056)
  • Fix streaming hang when retrying (CASSANDRA-9132)
  • Bind JMX to localhost unless explicitly configured otherwise (CASSANDRA-9085)
  • Add offline tool to relevel sstables (CASSANDRA-8301)
  • Preserve stream ID for more protocol errors (CASSANDRA-8848)
  • Fix combining token() function with multi-column relations on clustering columns (CASSANDRA-8797)
  • Fix hints serialized size calculation (CASSANDRA-8587)
  • Reject conditions on DELETE unless full PK is given (CASSANDRA-6430)
  • Properly reject the token function DELETE (CASSANDRA-7747)
  • Force batchlog replay before decommissioning a node (CASSANDRA-7446)

DataStax Enterprise 4.5.8 release notes 

Components
  • Cassandra 2.0.14.352
Changes and enhancements
  • Support for Java 8
Resolved issues
  • sstableloader pulling directory configs from cassandra.yaml causes failure (DSP-4630)
  • nodetool cleanup throws assertion as per CASSANDRA-8716 (DSP-4723)
  • Restarting a node after an abrupt shutdown might cause index data loss (DSP-4802)
  • Issuing "dsetool jobtracker" on non-Hadoop workload nodes causes an exception (DSP-4835)
  • Mahout demo fails to run when DSE_HOME is set for package installs (DSP-4837)
  • Set concurrent_compactors to 2 by default in cassandra.yaml (DSP-4849)
  • Fix Solr document expiration when row has already been physically deleted from Cassandra (DSP-4869)
  • A limitation of the current Sqoop import requires at least one value column (non-primary key) to do an import (DSP-5043)
  • Make updates replayed by the Solr commit log on startup immediately searchable (DSP-5095)
  • A system key created with AES/CBC/PKCS5Padding encryption cannot be initialized (DSP-5159)
  • Allow passing JMX username and password to DSE commands (DSP-5217)

Patches

  • Avoid race in cancelling compactions (CASSANDRA-9070)
  • More aggressive check for expired sstables in DTCS (CASSANDRA-8359)
  • Don't set clientMode to true when bulk-loading sstables to avoid a NullPointerException (CASSANDRA-8238)
  • Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
  • Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
  • java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
  • "disk full" when running cleanup (CASSANDRA-9036)
  • Make PasswordAuthenticator number of hashing rounds configurable (CASSANDRA-8085)
  • Lower logging level from ERROR to DEBUG when a scheduled schema pull cannot be completed due to a node being down (CASSANDRA-9032)
  • Fix MOVED_NODE client event (CASSANDRA-8516)
  • Allow overriding MAX_OUTSTANDING_REPLAY_COUNT (CASSANDRA-7533)
  • Fix malformed JMX ObjectName containing IPv6 addresses (CASSANDRA-9027)
  • Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949)
  • (cqlsh) Allow increasing CSV field size limit through cqlshrc config option (CASSANDRA-8934)
  • Stop logging range tombstones when exceeding the threshold (CASSANDRA-8559)
  • Fix NullPointerException when nodetool getendpoints is run against invalid keyspaces or tables (CASSANDRA-8950)
  • Allow specifying the tmp dir (CASSANDRA-7712)
  • Improve compaction estimated tasks estimation (CASSANDRA-8904)
  • Expose commit log archive status via JMX (CASSANDRA-8734)
  • Provide better exceptions for invalid replication strategy parameters (CASSANDRA-8909)
  • Fix regression in mixed single and multi-column relation support for SELECT statements (CASSANDRA-8613)
  • Add ability to limit number of native connections (CASSANDRA-8086)
  • Fix CQLSSTableWriter throwing exception and spawning threads (CASSANDRA-8808)
  • Fix MT mismatch between empty and GC-able data (CASSANDRA-8979)
  • Fix incorrect validation when snapshotting single table (CASSANDRA-8056)

DataStax Enterprise 4.5.7 release notes 

Components
  • Cassandra 2.0.12.201
Resolved issues
  • JMX defaults to binding only localhost to mitigate "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." After you upgrade to DataStax Enterprise 4.5.7, get the latest default values from cassandra-env.sh and ensure that your local cassandra-env.sh file has the latest default values. If you require remote access of JMX, you can safely enable remote JMX access by turning on JMX security. The latest default values from cassandra-env.sh are:
    LOCAL_JMX=yes
    if [ "$LOCAL_JMX" = "yes" ]; then  
        JVM_OPTS="$JVM_OPTS -Dcassandra.jmx.local.port=$JMX_PORT -XX:+DisableExplicitGC"
    else  
        JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"  
        JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"  
        JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false"  
        JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=true"  
        JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password"
    fi

DataStax Enterprise 4.5.6 release notes 

DataStax Enterprise 4.5.6 includes resolved issues, changes and enhancements, component changes, and patches.

Known issues
  • DataStax Enterprise 4.5.6 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Resolved issues

  • Dropping a Solr table leaves data behind in solr_admin (DSP-4068)
  • Package installer puts demos in different directory than the standalone installer (DSP-4644)
  • Search fails to start up with "Cannot find core" (DSP-4662)
  • SqlFreeFormQueryTest fails with StackOverflowError (DSP-4676)
  • Race condition while loading secondary indexes upon upgrade (DSP-4689)
  • Error while handling exceptions during netty-based shard connections (DSP-4754)
  • Explicitly set inter_dc_stream_throughput_outbound_megabits_per_sec because of CASSANDRA-8708 (DSP-4778)

Changes and enhancements

  • Implement sparse offheap filter to improve Solr pivot facet query performance (DSP-4518)

Component changes

  • Cassandra 2.0.12.200
  • Solr 4.6.0.2.10
  • Sqoop 1.4.5.15.1

Patches

 • Fix IllegalArgumentException in dynamic snitch (CASSANDRA-8448)
 • Add support for UPDATE ... IF EXISTS (CASSANDRA-8610)
 • Fix reversal of list prepends (CASSANDRA-8733)
 • Prevent non-zero default_time_to_live on tables with counters (CASSANDRA-8678)
 • Fix SSTableSimpleUnsortedWriter ConcurrentModificationException (CASSANDRA-8619)
 • Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
 • Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
 • Fix isClientMode check in Keyspace (CASSANDRA-8687)

DataStax Enterprise 4.5.5 release notes 

DataStax Enterprise 4.5.5 includes resolved issues, changes and enhancements, component changes, and patches.

Known issues
  • DataStax Enterprise 4.5.5 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Resolved issues

  • Sqoop cql-import fails because the Java driver uses the wrong data center. (DSP-4324)
  • The commons-io.jar in Sqoop library is incompatible with Hadoop. (DSP-4487)
  • Stress write can crash nodes. (DSP-4533)
  • DataStax Enterprise Sqoop from Oracle seems to send incorrect SQL syntax. (DSP-4579)
  • Indexing is disabled after race condition and causes double trigger registration. (DSP-4594)
  • Upgrade from 4.0.5 and earlier loses metadata. (DSP-4615)
  • Exceptions are generated when trying to access Cassandra from Spark REPL. (DSP-4629)

Changes and enhancements

  • Permissions are changed so that the cassandra user owns and manages DataStax Enterprise. (DSP-3585)
  • You must use the hive-metastore-migrate tool to map custom external tables to the new release format after upgrading to DataStax Enterprise 4.5.5. (DSP-4501)
  • Implement sparse offheap filter to enhance performance. (DSP-4518)

Component changes

  • Cassandra 2.0.12.156
  • Cassandra Java Driver 2.0.8. (DSP-4324)
  • Spark Cassandra Connector 1.0.6. (DSP-4603)

Patches

 • Use more efficient slice size for querying internal secondary
   index tables (CASSANDRA-8550)
 • Fix potentially returning deleted rows with range tombstone (CASSANDRA-8558)
 • Make sure we unmark compacting after scrub/cleanup etc (CASSANDRA-8548)
 • Check for available disk space before starting a compaction (CASSANDRA-8562)
 • Fix DISTINCT queries with LIMITs or paging when some partitions
   contain only tombstones (CASSANDRA-8490)
 • Introduce background cache refreshing to permissions cache
   (CASSANDRA-8194)
 • Fix race condition in StreamTransferTask that could lead to
   infinite loops and premature sstable deletion (CASSANDRA-7704)
 • Add an extra version check to MigrationTask (CASSANDRA-8462)
 • Ensure SSTableWriter cleans up properly after failure (CASSANDRA-8499)
 • Increase bf true positive count on key cache hit (CASSANDRA-8525)
 • Move MeteredFlusher to its own thread (CASSANDRA-8485)
 • Fix non-distinct results in DISTNCT queries on static columns when
   paging is enabled (CASSANDRA-8087)
 • Move all hints related tasks to hints internal executor (CASSANDRA-8285)
 • Fix paging for multi-partition IN queries (CASSANDRA-8408)
 • Fix MOVED_NODE topology event never being emitted when a node
   moves its token (CASSANDRA-8373)
 • Fix validation of indexes in COMPACT tables (CASSANDRA-8156)
 • Avoid StackOverflowError when a large list of IN values
   is used for a clustering column (CASSANDRA-8410)
 • Fix NPE when writetime() or ttl() calls are wrapped by
   another function call (CASSANDRA-8451)
 • Fix NPE after dropping a keyspace (CASSANDRA-8332)
 • Fix error message on read repair timeouts (CASSANDRA-7947)
 • Default DTCS base_time_seconds changed to 60 (CASSANDRA-8417)
 • Refuse Paxos operation with more than one pending endpoint (CASSANDRA-8346, 8640)
 • Throw correct exception when trying to bind a keyspace or table
   name (CASSANDRA-6952)
 • Make HHOM.compact synchronized (CASSANDRA-8416)

DataStax Enterprise 4.5.4 release notes 

The DSENRTCachingDirectoryFactory is deprecated. Modify the solrconfig.xml file to use the StandardDirectoryFactory instead of the DSENRTCachingDirectoryFactory.

Known issues
  • DataStax Enterprise 4.5.4 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Resolved issues

DataStax Enterprise 4.5.4 fixes these issues:
  • Fixed the issue preventing Hive from accessing uppercase keyspaces created using the Cassandra CLI. (DSP-4361)

DataStax Enterprise 4.5.3 release notes 

DataStax Enterprise 4.5.3 includes component changes, enhancements, changes, and resolved issues.

Known issues
  • DataStax Enterprise 4.5.3 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Component changes

DataStax Enterprise changes a few of the components introduced in the previous release. Changes are limited to these components:

  • Apache Cassandra 2.0.11.82
  • Apache Solr 4.6.0.2.8
  • Shark 0.9.1.4
  • Cassandra Java Driver 2.0.6
  • Spark connector 1.0.3

Enhancements and changes

  • The default Kerberos authenticator now supports the Digest authentication.
  • The spark.cleaner.ttl setting is configurable in the spark-env file, and set to 12 hours by default.
  • DSE Search now logs a warning when users do not map dynamic fields correctly.
  • Upgraded the Spark connector from 1.0.0 to 1.0.3, which includes the following enhancements:
    • 1.0.3
      • Fixed handling of Cassandra rpc_address set to 0.0.0.0 (#332)
    • 1.0.2
      • Fixed batch counter columns updates (#234, #316)
      • Expose both rpc addresses and local addresses of cassandra nodes in partition preferred locations (#325)

      • Cleaned up the SBT assembly task and added build documentation (backport of #315)

    • 1.0.1
      • Added logging of error message when asynchronous task fails in AsyncExecutor. (#265)

      • Fixed connection problems with fetching token ranges from hosts with rpc_address different than listen_address. Log host address(es) and ports on connection failures. Close thrift transport if connection fails for some reason after opening the transport, for example, authentication failure.

      • Upgraded cassandra driver to 2.0.6.

Resolved Issues

  • Fixed an issue causing Solr Join queries to return incorrect results. (DSP-3825)
  • Active commit log segments are now archived. (DSP-3873, CASSANDRA-6904)
  • Problems using S3 as an external file system have been resolved. (DSP-4082)
  • Fixed a problem causing an out-of-memory condition when running nodetool repair. (DSP-4104, CASSANDRA-7983)
  • Fixed an issue that prevented Hive from working with Kerberos if the user is not initialized on the node. (DSP-4127)
  • Fixed a null pointer exception during node bootstrap caused by the Solr core not being active yet when the back pressure kicks in. (DSP-4131)
  • Resolved the CQL TRUNCATE command problem related to memory-only tables. (DSP-4135)
  • Fixed an issue causing Shark to silently drop inserts. (DSP-4148)
  • The DataStax Enterprise integration of Spark respects the SPARK_DRIVER_HOST environment variable on start up. (DSP-4166)
  • The GossipFilePropertySnitch and EC2MultiRegionSnitch now use private IP addressses for communication during node repair between nodes in different data centers. (DSP-4192, CASSANDRA-8084)
  • Fixed bug causing core creation to fail if the Cassandra timeuuid type is used inside a list, set, or map collection. (DSP-4288)
  • Querying partitions directly now works. (DSP-4293)
  • Fixed an issue causing a null pointer exception on non Solr workload nodes holding Solr data and attempting to run the nodetool cleanup command on data. (DSP-4310)

DataStax Enterprise 4.5.2 release notes 

DataStax Enterprise 4.5.2 includes component changes, enhancements, changes, resolved issues, patches, and issues. The following issue resolved in this release might break existing code:

Possible code breaker

Hive now correctly returns NULL in places it was previously returning an empty string.

If your application inserts null string-type values into a table, adjust the code to expect NULL instead of an empty string.

Known issues
  • DataStax Enterprise 4.5.2 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Components

  • Apache Cassandra 2.0.10.71
  • Apache Hadoop 1.0.4.13
  • Apache Hive 0.12.0.5
  • Apache Pig 0.10.1
  • Apache Solr 4.6.0.2.6
  • Apache log4j 1.2.16
  • Apache Sqoop 1.4.4.14.2
  • Apache Mahout 0.8
  • Apache Tomcat 6.0.39
  • Apache Thrift 0.7.0
  • Apache Commons
  • Spark 0.9.1
  • Shark 0.9.1.3
  • JBCrypt 0.3m
  • SLF4J 1.7.2
  • Guava 15.0
  • JournalIO 1.4.2
  • Netty 4.0.13.Final
  • Faster XML 3.1.3
  • HdrHistogram 1.2.1.1
  • Snappy 1.0.5
  • Cassandra Java Driver 2.0.4.1
Enhancements and changes
  • Miscellaneous enhancements
    • The procedure for encrypting data has changed in this release. DataStax recommends migrating data encrypted in earlier releases to DataStax Enterprise 4.5.2.
    • This release supports spatial analytics through the integration of some components of GIS Tools for Hadoop. Also, included is a custom tool for importing data in Enclosed JSON format from ArcGIS to a Cassandra table.
    • An example of using Cassandra triggers in the demos directory.
    • You can now query the system.local and system.peers tables to get the type of workload a node is running.
    • A new merge metrics mbean provides information about Solr/Lucene segment merging.
  • Shark and Spark enhancements and changes
    • DataStax Enterprise Spark integration now uses the Spark Cassandra Connector. The package name has changed to com.datastax.spark.connector. You can use the configuration options defined in the project to configure DataStax Enterprise Spark.
    • During the upgrade to DataStax Enterprise 4.5.2, do not run Spark jobs until the entire cluster is upgraded to the new version of DataStax Enterprise.
    • As a workaround to https://spark-project.atlassian.net/browse/SHARK-217, this release updates the local Kyro-Serilization jar from 2.21 to 2.24. This fixes select statements having null fields.
    • In a file, you can specify a CQL schema for the Cassandra context when starting up Spark.
  • Sqoop enhancements
    • This release supports a native Cassandra implementation of the Sqoop metastore.
    • New cql-export options select certain columns for export, limit the page size of the export, and conditionally filter the data you select for export using the --cassandra-where-clause option <clause>.
    • In the previous release, you could not configure the node consistency level using the CQL import command. The --cassandra-consistency-level property can now be used for imports as well as exports.
  • Pig changes
    • Cql Paging Recorder Reader has been removed from Cassandra. The CqlStorage handler is no longer compatible with compact tables with clustering columns. Users who are accessing these tables need to migrate the tables to CqlNativeStorage format. This format uses the Cql Record Reader.
    • The CqlStorage handler is deprecated and slated for removal at some point in the future. Use the CqlNativeStorage handler and cql:// for new pig applications.
Resolved issues
  • CREATE TRIGGER and DROP TRIGGER now show up in the audit log. (DSP-2442)
  • This release of DSE Search/Solr compensates for the lack of scheduled hard commits to update the opened reader with the latest segments version. (DSP-3031)
  • Solr blob data types used in composite keys threw an invalidating error. (3033)
  • Incorrect authentication/authorization settings in Cassandra.yaml left cassandra in a state that prevented startup while keeping port 7199 closed. (DSP-3498)
  • Hive returned NULL in places it previously returned an empty string. See Possible code breaker. (DSP-3534)
  • The TermVectorComponent, not fixed in DSP-3147, caused unnecessary Cassandra access in CassandraRowReader due to the StoredFieldVisitor returning an empty collection of fields to load. (DSP-3625)
  • The problem that prevented repair, addition, and removal of nodes from a cluster using transparent data encryption has been resolved. When attempting to repair a node, the node could not unencrypt the data for reading. (DSP-3636).
  • The sstableloader utility works properly with encrypted tables when internal authentication, SSL, or kerberos authentication is used. (DSP-3636)
  • Improper loading of SSTables with encrypted tables using sstableloader when internal authentication, SSL, or kerberos authentication. (DSP-3641)
  • The null pointer exception no longer occurs when using QueryBuilder.batch from the Java driver and turning on auditing. (DSP-3673)
  • Incorrect names for options in the import.options file installed by DataStax Enterprise for the Sqoop demo have been fixed. (DSP-3708)
  • When installing in text mode, the installer failed to remove old dse JAR files. (DSP-3728)
  • DSE Search now uses the Tomcat HTTP NIO connector by default, unless otherwise specified in the server.xml. (DSP-3746)
  • Fixed an issue causing the Spark REPL to shutdown after a failure. (DSP-3795)
  • Fixed a memory leak in CFS compaction. (DSP-3799)
  • Fixed an Sqoop issue causing a null pointer exception to occur if not all of the cql columns were specified in the cql-export command. (DSP-3803)
  • Fixed an issue causing Solr Join queries to return incorrect results. (DSP-3825)
  • Previously NULL values were returned as empty datatypes in hive. Now, the exact information from the native protocol is returned, except collections, which are returned as empty collections when the value is NULL. (DSP-3840)
  • Fixed an issue causing certain DSE tools to warn about an incorrect configuration loader. (DSP-3842)
  • Fixed an issue, which occurred in specific situations if an analytic node is down, that prevented Hive from accessing CFS. (DSP-3846)
  • Fixed the issue causing Spark Master HA to fail to start sometimes on a single node. (DSP-3855)
  • Fixed the build.xml file that ships with the Spark demo. (DSP-3886)

Patches

  • Better error message when adding a collection with the same name than a previously dropped one (CASSANDRA-6276)
  • Pig support for hadoop CqlInputFormat (CASSANDRA-6454)
  • Add inter_dc_stream_throughput_outbound_megabits_per_sec (CASSANDRA-6596)
  • Fix potential AssertionError with 2ndary indexes (CASSANDRA-6612)
  • Add option to disable STCS in L0 (CASSANDRA-6621)
  • Hadoop--Add CqlOutputFormat (CASSANDRA-6927)
  • Always merge ranges owned by a single node (CASSANDRA-6930)
  • cqlsh--Wait up to 10 sec for a tracing session (CASSANDRA-7222)
  • Give CRR a default input_cql Statement (CASSANDRA-7226)
  • Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229)
  • Hadoop--allow ACFRW to limit nodes to local DC (CASSANDRA-7252)
  • Workaround JVM NPE on JMX bind failure (CASSANDRA-7254)
  • Fix race in FileCacheService RemovalListener (CASSANDRA-7278)
  • Fix inconsistent use of consistencyForCommit that allowed LOCAL_QUORUM operations to incorrect become full QUORUM (CASSANDRA-7345)
  • Make sure high level sstables get compacted (CASSANDRA-7414)
  • Properly handle unrecognized opcodes and flags (CASSANDRA-7440)
  • Fix AssertionError when using empty clustering columns and static columns (CASSANDRA-7455)
  • Hadoop--close CqlRecordWriter clients when finished (CASSANDRA-7459)
  • Switch liveRatio-related log messages to DEBUG (CASSANDRA-7467)
  • Set correct stream ID on responses when non-Exception Throwables are thrown while handling native protocol messages (CASSANDRA-7470)
  • Remove duplicates from StorageService.getJoiningNodes (CASSANDRA-7478)
  • Fix error when doing reversed queries with static columns (CASSANDRA-7490)
  • Properly reject operations on list index with conditions (CASSANDRA-7499)
  • Throw InvalidRequestException when queries contain relations on entire collection columns (CASSANDRA-7506)
  • Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
  • Fix truncate to always flush (CASSANDRA-7511)
  • Warn when SSL certificates have expired (CASSANDRA-7528)
  • Fix range merging when DES scores are zero (CASSANDRA-7535)
  • Windows--force range-based repair to non-sequential mode (CASSANDRA-7541)
  • Fix row size miscalculation in LazilyCompactedRow (CASSANDRA-7543)
  • Backport CASSNADRA-3569/CASSANDRA-6747 (CASSANDRA-7560)
  • Remove CqlPagingRecordReader/CqlPagingInputFormat (CASSANDRA-7570)
  • Fix ReversedType aka DateType mapping to native protocol (CASSANDRA-7576)
  • cqlsh--enable CTRL-R history search with libedit (CASSANDRA-7577)
  • Fix sstableloader unable to connect encrypted node (CASSANDRA-7585)
  • Add stop method to EmbeddedCassandraService (CASSANDRA-7595)
  • Remove shuffle and taketoken (CASSANDRA-7601)
  • cqlsh--Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611)
  • Update java driver for hadoop (CASSANDRA-7618)
  • Fix NPE when listing saved caches dir (CASSANDRA-7632)
  • Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635)
  • cqlsh--cqlsh should automatically disable tracing when selecting from system_traces (CASSANDRA-7641)
  • Track max/min timestamps for range tombstones (CASSANDRA-7647)
  • Add cassandra.auto_bootstrap system property (CASSANDRA-7650)
  • SimpleSeedProvider no longer caches seeds forever (CASSANDRA-7663)
  • Throw EOFException if we run out of chunks in compressed datafile (CASSANDRA-7664)
  • Set gc_grace_seconds to seven days for system schema tables (CASSANDRA-7668)
  • Support connecting to ipv6 jmx with nodetool (CASSANDRA-7669)
  • Avoid logging CompactionInterrupted at ERROR (CASSANDRA-7694)
  • Fix potential AssertionError in RangeTombstoneList (CASSANDRA-7700)
  • cqlsh--Fix failing cqlsh formatting tests (CASSANDRA-7703)
  • Validate arguments of blobAs functions (CASSANDRA-7707)
  • Minor leak in sstable2jon (CASSANDRA-7709)
  • Fix validation when adding static columns (CASSANDRA-7730)
  • Thrift--fix range deletion of supercolumns (CASSANDRA-7733)
  • Fix dropping collection when it's the last regular column (CASSANDRA-7744)
  • Fix race in background compaction check (CASSANDRA-7745)
  • Do not flush on truncate if durable_writes is false (CASSANDRA-7750)
  • Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752)
  • Configure system.paxos with LeveledCompactionStrategy (CASSANDRA-7753)
  • Fix NPE in FileCacheService.sizeInBytes (CASSANDRA-7756)
  • Clone token map outside of hot gossip loops (CASSANDRA-7758)
  • Hadoop--fix cluster initialisation for a split fetching (CASSANDRA-7774)
  • Fix PRSI handling of CQL3 row markers for row cleanup (CASSANDRA-7787)
  • Improve PasswordAuthenticator default super user setup (CASSANDRA-7788)
  • Make StreamReceiveTask thread safe and gc friendly (CASSANDRA-7795)
  • Stop inheriting liveRatio and liveRatioComputedAt from previous memtables (CASSANDRA-7796)
  • Validate empty cell names from counter updates (CASSANDRA-7798)
  • Fix ALTER clustering column type from DateType to TimestampType when using DESC clustering order (CASSANRDA-7797)

Issues

  • In this release, compaction of files stored on the cfs-archive layer should be disabled, but instead these files are compacted automatically. (DSP-4081)

DataStax Enterprise 4.5.1 release notes 

Known issues
  • DataStax Enterprise 4.5.1 and earlier releases are affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Components

  • Apache Solr 4.6.0.2.4 to 4.6.0.2.5
  • Shark 0.9.1.1 to 0.9.1.2
  • Cassandra Java Driver 2.0.2 to 2.0.2.1
Enhancements and changes
  • Post Solr query parameters are now properly stored in the audit logs.
  • This release adds the update metrics mbean, which can be useful to guide tuning of all factors affecting indexing performance, such as back pressure, indexing threads, RAM buffer size and merge factor.
  • The Weather Sensor demo for running analytical queries with Hadoop and Spark is now easier to run. You no longer need to change path variables in tarball installations.
  • The Weather Sensor demo readme file has been improved.
  • The Shark component, updated to 0.9.1.2, works with internal authentication.
  • The showSchema method, which has been added to Spark, provides information about all user keyspaces, a particular keyspace, or a table.
  • Improved default memory settings for Spark.
  • When running the Portfolio Manager demo, messages about keyspace sensitivity no longer appear.

Resolved issues

  • Resolved the issue causing DataStax Enterprise to hang during shutdown, waiting for gossip to start. (DSP-3518)
  • Fixed the out-of-memory error on huge clusters caused by Cassandra File System (CFS) memory consumption, which has been reduced significantly, approximately 500 times for some use cases. (DSP-3615)
  • Fixed an issue when enabling clustering in Solr that caused DataStax Enterprise to complain about the org.tartarus.snowball.ext.DanishStemmer class not being found. (DSP-3645)
  • Fixed the race condition between shutdown of client and server channels, causing a harmless Netty exception to appears when you shut down. (DSP-3651)
  • Fixed the problem preventing the HSHA rpc server from functioning. (DSP-3675)
  • RPM and Deb package installations now properly find the shark-env.sh file. (DSP-3696)

DataStax Enterprise 4.5 release notes 

DataStax Enterprise 4.5 updates components and includes major enhancements improvements, patches, and bug fixes.

Known issues
  • DataStax Enterprise 4.5 is affected by "CVE-2015-0225: Apache Cassandra remote execution of arbitrary code." Upgrade to the latest release or enable JMX security immediately if your JMX port is possibly exposed to malicious users.

Components

  • Apache Cassandra 2.0.8.39
  • Apache Hadoop 1.0.4.13
  • Apache Hive 0.12.0.3
  • Apache Pig 0.10.1
  • Apache Solr 4.6.0.2.4
  • Apache log4j 1.2.16
  • Apache Sqoop 1.4.4.14.1
  • Apache Mahout 0.8
  • Apache Tomcat 6.0.39
  • Apache Thrift 0.7.0
  • Apache Commons
  • Spark 0.9.1
  • Shark 0.9.1.1
  • JBCrypt 0.3m
  • SLF4J 1.7.2
  • Guava 15.0
  • JournalIO 1.4.2
  • Netty 4.0.13.Final
  • Faster XML 3.1.3
  • HdrHistogram 1.0.9
  • Snappy 1.0.5
  • Cassandra Java Driver 2.0.2
Apache Cassandra documentation covers release notes for Cassandra 2.0.8. NEWS.txt contains late-breaking information about upgrading from previous versions of Cassandra. A NEWS.txt or a NEWS.txt archive is installed in the following locations:
  • Tarball: install_location/resources/cassandra
  • Installer-Services installations: /usr/share/dse/resources/cassandra
  • Package: /usr/share/doc/dse-libcassandra*

NEWS.txt is also posted on the Apache Cassandra project web site.

Enhancements and changes

DataStax Enterprise 4.5 includes the following enhancements and changes:
  • Spark/Shark
    • Support for Apache Spark for running analytical queries independent of Hadoop
    • Support for Apache Shark, a SQL-like, Hive-compatible language built on top of Spark
  • External Hadoop systems
    • A bring your own Hadoop (BYOH) model that integrates Hadoop data warehouse implementations by Cloudera and Hortonworks
    • Support for Kerberos-secured BYOH integration using the Cloudera Manager
  • DSE Hadoop/Hive/Pig
    • Support for the native protocol in Hive including the addition of 19 new Hive TBLPROPERTIES to support the native protocol
    • Auto-creation of Hive databases and external tables for each CQL keyspace and table
    • A new cql3.partition.key property that maps Hive tables to CQL compound primary keys and composite partition keys
    • Support for HiveServer2
    • Integration of the HiveServer2 Beeline command shell
    • Support for expiring data in columns by setting TTL (time to live) on Hive tables.
    • Support for expiring data by setting the TTL on Pig data using the cql:// URL, which includes a prepared statement shown in step 10 of the library demo.
  • Sqoop
  • Solr
    • For performance, you can configure DSE Search/Solr to parallelize row reads.
    • The default shard transport type has been changed from http to netty. If you upgrade to DataStax Enterprise 4.5, perform the upgrade procedure using the shard transport type of your old installation, and after the upgrade, change the shard transport type to netty. Start the cluster using a rolling restart.
    • This release of DataStax Enterprise does not use Lucene compressed stored fields anymore for performance reasons. Subsequent releases will also not use these fields. (DSP-3484)
    • When the Solr query time join from field is docValues=true, the faster doc values-based join system is used. Upgrading to DataStax Enterprise 4.5 requires reindexing if query time join is used.
    • DataStax Enterprise 4.5 and later moves the DSE per-segment filter cache off-heap by using native memory, hence reducing on-heap memory consumption and garbage collection overhead.
    • The new off-heap filter cache is enabled by default, but can be disabled by passing the following JVM system property at startup time: -Dsolr.offheap.enable=false.
    • Query metric times are now logged by DSE Search.
    • DSE mbean names have been improved to decrease the chance that names will clash. The old naming format was com.datastax.bdp:type=name. The new format is com.datastax.bdp:type=workload,name=name. For example, com.datastax.bdp:type=search,name=SolrContainerPlugin.

Resolved issues

  • Solr
    • Soft commit adjustments during back pressure are now correctly executed. (DSP-3584)
    • The problem that caused the old pre-reload value of the soft commit to be reloaded during back pressure has been resolved. (DSP-3584)
    • Resolved a problem causing Tomcat to block shutdown when an out of memory error occurs. (DSP-3328)
  • Other
    • Resolved a problem caused by the disk full alert feature that removed a node from the ring. Because Cassandra will no longer automatically decommission a node when the disk is almost full, you need to monitor disk space and add capacity when necessary. (DSP-3601)
    • The DSE init.d script sets -XX:HeapDumpPath when using jsvc fallback and also when using the default dse_daemon script. The latter was not being set, which prevented the heap from being dumped on an out-of-memory condition. (DSP-3308)
    • Resolved a problem caused by calling Gossiper.instance.getEndpointStateForEndpoint but not checking for a null return, which lead to null pointer exceptions like the following (DSP-3616):
      ERROR 11:02:42,841 Exception in thread Thread[Thread-1,5,main]
      java.lang.NullPointerException
      at com.datastax.bdp.gms.DseState.doGetCurrentState(DseState.java:269)
      at com.datastax.bdp.gms.DseState.setActiveStatus(DseState.java:167)
      at com.datastax.bdp.server.DseDaemon.stop(DseDaemon.java:470)
      at com.datastax.bdp.server.DseDaemon$1.run(DseDaemon.java:380)

Issues

  • DataStax supports a data center that contains one or more nodes running in dual Spark/DSE Hadoop mode. DataStax does not support running some nodes in DSE Hadoop mode and some in Spark mode in the same data center. Dual Spark/DSE Hadoop mode means you started the node using the -k and -t options on tarball installations, or set the startup options HADOOP_ENABLED=1 and SPARK_ENABLED=1 on packaged installations. (DSP-3561)
  • Due to a DSE_CLASSPATH problem, if you are installing DataStax Enterprise to use the bring your own Hadoop (BYOH) model, you need to install and configure DataStax Enterprise on all nodes, including nodes in the Hadoop cluster, as described in the installation procedure. (DSP-3654)
  • Due to a race condition between shutdown of client and server channels, a harmless Netty exception appears when you shut down. The exception looks something like this:
    WARN 20:18:15,397 Failed to submit an exceptionCaught() event.
    java.util.concurrent.RejectedExecutionException: event executor terminated
    at io.netty.util.concurrent.SingleThreadEventExecutor.reject
    (SingleThreadEventExecutor.java:703) . . .

    Ignore this exception. (DSP-3651)

  • In this release, the HSHA rpc server is not functioning. (DSP-3675)

  • Writing to Blob columns from Spark is not supported in this release. Reading columns of all types is supported; however, you need to convert collections of blobs to byte arrays before serializing. (DSP-3620)
  • After upgrading DataStax Enterprise from 4.0.0 or 4.0.1 to 4.5.x on RHEL5/CentOS5, the Snappy JAR file will be missing. To get it back, either:
    • Run the switch-snappy script:
      $ cd /usr/share/dse ## Package installations
      $ cd install_location/bin  ## tarball installations
      
      $ switch-snappy 1.0.4
    • Uninstall the old installation and then do a fresh installation. Using, a regular uninstall maintains the configuration files and data files.
  • Cassandra static columns, introduced in Cassandra 2.0.6, cannot be included in the Solr schema (and hence indexed) for performance reasons because changing the value of a single static column would require re-indexing all documents sharing the same partition key. (DSP-3143)

DataStax Enterprise 4.5 Installer release notes 

Components

  • Apache Cassandra 2.0.8
  • OpsCenter 4.1.4
  • DevCenter 1.1.x