DataStax Enterprise 6.7 release notes

8 September 2020

6.7.10 Components

DSE 6.7.10 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

6.7.10 DSE database

Changes and enhancements:

• Adds TTL and TimeWindowCompactionStrategy (TWCS) to system_distributed.repair_history and system_distributed.parent_repair_history tables. (DB-2009)
• Remove a race condition that may lead to reopening a keyspace during keyspace drop. (DB-4564)
• Workaround for LOGBACK-1194 - explicit scanPeriod added to logback.xml. (DSP-17911)
• After sufficiently long node uptime encrypted index will write no more data (DSP-21234)

6.7.10 DSE security

6.7.10 DSEFS

6.7.10 DSE Search

6.7.10 DSE Advanced Replication

Known issues: None

6.7.10 DSE Graph

Changes and enhancements:

None.

6.7.10 DSE Analytics

DataStax Studio

• See .

DataStax Bulk Loader

• See DataStax Bulk Loader release notes.

DataStax Apache Kafka™ Connector

• See DataStax Apache Kafka Connector release notes.

23 July 2020

6.7.9 Components

DSE 6.7.9 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.9 highlights

DSE database

New systemd units for package installs on RHEL and other compatible operating systems. See .

DSE security

Enhanced LDAP settings to properly handle nested groups so that LDAP enumerates all ancestors of a user's distinguishedName. Inherited groups retrieval with directory_search and members_search types. Fixed fetching parent groups of a role that's mapped to an LDAP group. See the security section of this topic.

6.7.9 DSE database

Changes and enhancements:

• New systemd units for package installs on RHEL and other compatible operating systems. See . (DSP-7603)
• New snapshot_before_dropping_column configuration option added in cassandra.yaml. See snapshot_before_dropping_column.(DB-2690)
• Increased the default direct_reads_size_in_mb value in cassandra.yaml. See direct_reads_size_in_mb. (DB-4348)
• New entries to jvm.options to assist with capturing thread dumps. (DSP-20778)
• PKCS11 keystore is supported in Advanced workloads. See keystore_type. (DSP-20094)
• Added entries to jvm.options to assist with capturing thread dumps. (DSP-20778)
• The frequency of range queries performed by lease manager is now configurable via the new dse.lease.refresh.interval.seconds system property. The new option is in addition to configuration via JMX and dsetool, added in DSE 6.7.8. (DSP-20696)
• CQLSH can be run with Python 2 or 3. The Python 3 support is new. (DB-4151)
• When working on encrypted SSTables, Cassandra tools such as sstabledump or sstablesplit need to create a remote connection in ReplicatedKeyProvider to a Cassandra node in order to be able to decrypt the sstables. (The sstableloader tool is not affected.) However, if security is enabled on the cluster, the tools won't be able to decrypt the sstables because they have no way of specifying security parameters for the cluster connection. (DSP-12666)
• LDAP servers can now handle multiple, comma separated addresses, with or without a port. If the port is not provided, the ldap_options.server_port parameter is used by default. See LDAP options. (DSP-13086)

6.7.9 DSE security

6.7.9 DSEFS

6.7.9 DSE Search

6.7.9 DSE Advanced Replication

Known issues: None

6.7.9 DSE Graph

Changes and enhancements:

6.7.9 DSE Analytics

DataStax Studio

• See .

DataStax Bulk Loader

• See DataStax Bulk Loader release notes.

DataStax Apache Kafka™ Connector

• See DataStax Apache Kafka Connector release notes.

4 May 2020

6.7.8 Components

DSE 6.7.8 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.8 Highlights

DSE Search

6.7.8 DSE core

Changes and enhancements:

• dsetool ring shows in-progress search index building during bootstrap. (DSP-15281)
• Added dse.ldap.retry_interval.ms system property, which sets the time between subsequent retries when trying authentication using LDAP server. (DSP-20298)
• Removed Jodd Core dependency that created vulnerability to Arbitrary File Writes. (DSP-19206)
• Made the search reference visible in the error message for LDAP connections. (DSP-20578)
• DecayingEstimatedHistogram now decays even when there are no updates so invalid metric values do not linger. (DSP-20674)
• Added hostname_verification to ldap_options in dse.yaml. (DSP-20302)
• Added a new JMX attribute of ConnectionSearchPassword for LdapAuthenticator bean has been added, which updates the LDAP search password without the need to restart DSE. (DSP-18928)
• The frequency of range queries performed by lease manager is now configurable via JMX and dsetool command. (DSP-20696)
• Added functionality to query role_stats when stats is enabled under role_management_options in dse.yaml. (DB-4283)
• Fixed bug to allow LIST ROLES and LIST USERS to work with system-keyspace-filtering enabled. (DB-4221)
• The replica side filtering dtests test_update_on_wide_table and test_complementary_update_with_limit_on_static_column_with_not_empty_partitions are more reliable. (DB-4043)
• Nodesync can now be enabled on all system distributed and protected tables. (DB-3241)
• Improved the estimated values of histogram percentiles reported via JMX. In some cases, the percentiles may go slightly up for DSE 6.0. (DB-4275)
• Added anticompaction to nodetool stop command help menu. (DB-3821)
• Added --disable-history option to cqlsh that disables saving history to disk for current execution. Added history section to cqlshrc which is called with boolean parameter disabled that is set to False by default. (DB-3843)
• Improved output of nodetool tablestats to accurately report when no data is available. (DB-3854)
• Improved error messaging for enabled internode SSL encryption in Cassandra Tools test suite. (DB-3957)
• Removed the serialization header partition/clustering key validation (DB-4111)
• Security update:
  • Upgraded Jackson Core and Jackson Mapper to address CVE-2019-10172. (DSP-20073)

6.7.8 DSE Advanced Replication

6.7.8 DSE Analytics

Changes and enhancements:

• Removed Jodd Core dependency that created vulnerability to Arbitrary File Writes. (DSP-19206)
• Added spark.cassandra.session.consistency.level parameter to Spark Connector. Set HiveMetaStore default consistency level to LOCAL_QUORUM instead of ONE. (DSP-19982)
• During Spark Application startup, Exception: java.lang.ExceptionInInitializerError thrown from the UncaughtExceptionHandler in thread "main" was logged, sometimes instead of a meaningful error. (DSP-20474)
• Bring-Your-Own-Spark (BYOS) builds include dependencies for Joda and Commons-Configuration. (DSP-20512)
• Internal continuous paging sessions were not closed when LIMIT clause was added in SQL query, which caused sessions leak and inability to close the Spark application gracefully because the Java driver waited indefinitely for orphaned sessions to finish. (DSP-19804)
• Security update:
  • Patched hive with HIVE-13390 to fix CVE-2016-3083. (DSP-20612)

6.7.8 DSEFS

Changes and enhancements:

• DSEFS local file system implementation returns alphabetically sorted directories and files when using wildcards and listing command. (DSP-20057)
• When creating a file through WebHDFS API, DSEFS does not verify WX permissions of parent's parent when the parent exists. (DSP-20355)
• DSEFS internode encrypted communication doesn't fail when server_encryption_options.require_endpoint_verification is enabled. (DSP-20689)

Resolved issues:

• DSEFS cannot use Mixed Case keyspaces, which was broken by DSP-16825. (DSP-20354)

6.7.8 DSE Graph

Changes and enhancements:

• Exposed configuration and metrics for Gremlin query cache. (DSP-20240)
• Changed classic Graph query so vertices are read from _p tables in Cassandra using SELECT ... WHERE <vertex primary key columns statement. The search predicate is applied in memory. (DSP-20230)
• Fixed Graph cleanUp failures due to duplicate properties on dangling edges. (DSP-20460)

Resolved issues:

6.7.8 DSE Search

Changes and enhancements:

• Error messages related to Solr errors contain better description of the root cause. (DSP-13792)
• The dsetool stop_core_reindex command now mentions the node in the output message. (DSP-17090)
• Added indexing reason to output of dsetool core_indexing_status command. (DSP-17672)
• Improved warnings for search index creation via dsetool or CQL. (DSP-17994)
• Improved guidance with warnings when index rebuild is required for ALTER SEARCH INDEX, RELOAD SEARCH INDEX, and dsetool reload_core commands. (DSP-19347)
• Improved real-time search to fix a docValues bug. (DSP-20300)
• suggest request handler requires select permission. Previously, suggest request handler returned forbidden response when authorization was on, regardless of the user permissions. (DSP-20697)
• Security update:
  • Upgraded Apache Solr to address CVE-2018-8026. (DSP-16653)

10 December 2019

6.7.7 Components

DSE 6.7.7 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.7 Highlights

DSE Search

6.7.7 DSE core

Changes and enhancements:

• Add support for nodesync command to specify different IP addresses for JMX and CQL. (DB-2969)
• Dropping and re-adding the same column with incompatible types is not supported. This change prevents unreadable SSTables. (DB-3586)
• Enhancements to the offline sstablescrub utility. (DB-3510, DB-3511)
  • Specify which SSTables to scrub.
  • Scrub multiple tables of the same keyspace.
  • Specify the number of threads to simultaneously scrub SSTables within a table.
• Prevent accepting streamed SSTables or loading SSTables when the clustering order does not match. (DB-3530)
• RoleManager cache keeps invalid values if the LDAP connectivity is down. (DSP-20098)
• LDAP user login fails due to parsing failure on user DN with parentheses. (DSP-20106)

Resolved issues:

Known issues:

openssl pkcs12 -export -nokeys -out truststore.pfx -in intermediate.chain.pem

keytool -importcert -alias rootca -file rootca.pem -keystore truststore.jks

keytool -importcert -alias intermediate -file intermediate.pem -keystore truststore.jks

keytool -importkeystore -srckeystore truststore.jks -destkeystore truststore.pfx -deststoretype pkcs12 

6.7.7 DSE Analytics

Changes and enhancements:

• Add new DSE class com/datastax/bdp/spark/* for dse-spark-dependencies. (DSP-16070)
• Improve configuration of available system resources for Spark Workers. You can now set the total memory and total cores with new environment variables that take precedence over the resource_manager_options defined in dse.yaml. (DSP-19673)

  dse.yaml resource_manager_options	Environment variable
  memory_total	SPARK_WORKER_TOTAL_MEMORY
  cores_total	SPARK_WORKER_TOTAL_CORES

• Support for multiple contact points is added for DSEFS implementation of the Hadoop FileSystem. (DSP-19704)
  Provide FileSystem URI with:

  dsefs://host0\[:port\]\[,host1\[:port\]\]/

6.7.7 DSE Search

Changes and enhancements:

• The Solr timeAllowed option in queries is now enforced by default to prevent long-running shard queries. This change prevents complex facets and boolean queries from using system resources after the DSE Search coordinator considers the queries to have timed out. For all queries, the default for the timeAllowed value uses the value of client_request_timeout_seconds setting in dse.yaml. (DSP-19781, DSP-19790)
  Attention: While using Solr timeAllowed in queries improves performance for long zombie queries, it can cause increased per-request latency cost in mixed workloads. If the per-request latency cost is too high, use the -Ddse.timeAllowed.enabled.default search system property to disable timeAllowed in your queries.

Resolved issues:

• Apply filter cache optimization to remote shard requests when RF=N. (DSP-19800)
• Filter cache warming doesn't warm parent-only filter correctly when RF=N. (DSP-19802)
• Memory allocation issue causes performance degradation at query time. (DSP-19805)

22 October 2019

6.7.6 Components

DSE 6.7.6 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.6 Highlights

DSE Search

6.7.6 DSE core

Changes and enhancements:

• DSE version now appears as a comment in all configuration files. (DB-1022)
• Allocate large buffers directly in the chunk cache. (DB-3506)
• New nodetool commands to get current values: getcachecapacity, getcachekeystosave, and gethintedhandoffthrottlekb. (DB-3618)

Resolved issues:

• Background compactions block access to SSTable. (DB-3682)
• Post-bootstrap indexing is executed only by a single CPU core. (DB-3692)
• Reads against ma and mc SSTables hit more SSTables than necessary due to the bug fixed by CASSANDRA-14861. (DB-3691)
  Attention: DataStax recommends reading and following all upgrade instructions in Upgrade DataStax Enterprise documentation. To ensure that all SSTables are on the current version, upgrade the SSTables on each node. If the SSTables are already on the current version, the command returns immediately and no action is taken. See DataStax Enterprise, Apache Cassandra, CQL, and SSTable compatibility.
• Post-bootstrap indexing is executed by only a single CPU core. (DB-3692)
• Upgrade Jackson Databind to address CVE-2019-14540. (DSP-19764)

6.7.6 DSE Analytics

6.7.6 DSE Search

Changes and enhancements:

• For token ranges dictated by distribution, filter cache warming occurs when a node is restarted, a search index is rebuilt, or when node health score is up to 0.9. New per-core metrics for metric type WarmupMetrics and other improvements. (DSP-8621)

26 August 2019

6.7.5 Components

DSE 6.7.5 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.5 Highlights

High-value benefits of upgrading to DSE 6.7.5 include these highlights:

DSE Database (DSE core) highlights

• Fixed incorrect handling of frozen type issues to accept all valid CQL statements and reject all invalid CQL statements. (DB-3084)
• Standalone cqlsh client tool provides an interface for developers to interact with the database and issue CQL commands without having to install the database software. From DataStax Downloads, download the version of CQLSH that corresponds to your DataStax database version. (DSP-18694)
• New options to select cipher suite and protocol to configure KMIP encryption when connecting to a KMIP server. (DSP-17294)

DSE Analytics highlights

• Storing and revoking permissions for the application owner is removed. The owner of the application is explicitly assumed to have these permissions. (DSP-19393)

DSE Search highlights

• Unbounded facet searches are no longer allowed. (DSP-18693)
  • facet.limit < 0 is no longer supported. Override the default facet.limit of 20000 with the -Dsolr.max.facet.limit.size system property.
  • This change adds guardrails that can cause misconfigured faceting queries to fail. Before upgrading, set an explicit facet.limit.
• Improved DSE 6.7 search query latency. (DSP-18677)

6.7.5 DSE core

Changes and enhancements:

• Enhanced DroppedMessages logging output adds the size percentiles of the dropped messages, their most common destinations, and for read requests or mutations, the most common tables being targeted. (DB-1250)
• Improved troubleshooting. A log entry is now created when autocompaction is disabled or enabled for a table. (DB-1635)
• Reformatted StatusLogger output to reduce details in the INFO-level system.log. The detailed output still exists in the debug.log. (DB-2552)
• When unable to send mutations to replicas due to overloading, hints are mistakenly created against the local node. (DB-3421)
• When a non-frozen UDT column is dropped and the table is later recreated from the schema that was created as part of a snapshot, the dropped column record is invalid and may lead to failure loading some SSTables. (DB-3434)
• New resources improve debugging leaked chunks before the cache evicts them and provide more meaningful call stack and stack trace. (DB-3504)
  • RandomAccessReader/RandomAccessReader
  • AsyncPartitionReader/FlowSource
  • AsyncSSTableScanner/FlowSource
• New options to select cipher suite and protocol to configure KMIP encryption when connecting to a KMIP server. (DSP-17294)
• Standalone cqlsh client tool provides an interface for developers to interact with the database and issue CQL commands without having to install the database software. From DataStax Tools, download the version of CQLSH that corresponds to your DataStax database version. (DSP-18694)
• Error in custom provider prevents DSE node startup. With this fix, the node will start up but insights is not active. See the DataStax Support Knowledge Base for steps to resolve existing missing or incorrect keyspace replication problems. (DSP-19521)

Resolved issues:

• Fix to prevent NPE during repair in mixed-version clusters. (DB-1985)
• Prevent changing the replication strategy of system keyspaces. (DB-2960)
• Incorrect handling of frozen type issues: valid CQL statements are not accepted and invalid CQL statements are not property rejected. (DB-3084)
• sstablepartitions incorrectly handles -k and -x options. (DB-3443)

  Workaround: To specify multiple keys, repeat the -k or -x options several times.

• Memory leaks when updating tables with materialized views. (DB-3504)
• Buffers should return to the pool if a chunk is leaked. (DB-3512)
• sstablepartitions incorrectly handles -k and -x options. (DB-3442)

  Workaround: To specify multiple keys, repeat the -k or -x option for each key.

• Upgrade Jackson Databind to address CVE-2018-11307. (DB-2911, DSP-19319)
• Upgraded Apache MINA Core library to 2.0.21 to prevent a security issue where Apache MINA Core was vulnerable to information disclosure. (DSP-19213)
• Update Jackson Databind to 2.9.9.1 for all components except DataStax Bulk Loader. (DSP-19441)

6.7.5 DSE Analytics

6.7.5 DSE Graph

Changes and enhancements:

• New graph truncate command to remove all data from graph. (DSP-17609)
• Support for ifExists() before truncate(), like system.graph("foo").ifExists().truncate(), in DSE Graph (classic graph) API. (DSP-19357)

6.7.5 DSE Search

Resolved issues:

• Solr CQL count query incorrectly returns the count as all data count but should return all data count minus start offset. (DSP-16153)
• When driver uses paging, CQL query fails when using a Solr index to query with a sort on a field that contains the primary key name in the field: InvalidRequest: Error from server: code=2200 [Invalid query] message="Cursor functionality requires a sort containing a uniqueKey field tie breaker". (DSP-19210)

27 June 2019

6.7.4 Components

DSE 6.7.4 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.4 Highlights

High-value benefits of upgrading to DSE 6.7.4 include these highlights:

DSE Database (DSE core) highlights

• Significant fixes and improvements for native memory, the chunk cache, and async read timeouts.
• Improved logging identifies which client, keyspace, table, and partition key is rejected when mutation exceeds size threshold. (DB-1051)
• Improved lightweight transactions (LWT) handling. (DB-3018)
• Prevent hang during startup when reading counter cache. (DB-3050)
• New configurable memory leak tracking. (DB-3123)
• Fixed an issue by incrementing pending echos when sending gossip echo requests. (DB-3187)
• Fixed an issue that caused an error when using tarball installs to create two instances on the same physical server with remote JMX access. (DB-2483)
• Improved status reporting for nodesync validation list. (DB-2707)
• Bootstrap now fails when the node is not able to fetch the schema from other nodes in the cluster. (DB-3186)
• Fixed an issue to prevent deadlock when replaying schema mutations from commit log during DSE startup. (DB-3190)

DSE Analytics highlights

• When DSE authentication is enabled, Spark security is forced to be enabled. (DSP-17274)
• Spark security is turned on in dse.yaml configuration file. (DSP-17271)

DSE Graph highlights

• Improved gremlin-console startup time. (DSP-11550)
• Fixed an issue where DseGraphFrame cannot directly copy graph from one cluster to another. You can now dynamically pass cluster and connection configuration for different graph objects. (DSP-18605)
• Fixed an issue where UnsatisfiedLinkError occurs when insert multi edge with DseGraphFrame in BYOS (Bring Your Own Spark). (DSP-18916)
• DSE Graph does not use primary key predicate in Search/.has() predicate. (DSP-18993)
• Fixed an issue where T values get hidden by property keys of the same name in valueMap(). (DSP-19261)
• Fixed an issue where graph Solr queries might not be run on a TPC Scheduler. (DSP-18898)

DSE Search highlights

• Performance improvements and overload protection for search queries. (DSP-15875)
• Performance improvements to Solr deletes that correspond to Cassandra rows. (DSP-17419)
• Fixed an issue where FQ was broken with queryExecutorThreads and timeAllowed set. (DSP-18717)
• Changes to correct uneven distribution of shard requests with the STATIC set cover finder. (DSP-18197)
• New recommended method for case-insensitive text search, faceting, grouping, and sorting with new LowerCaseStrField Solr field type. This type sets field values as lowercase and stores them as lowercase in docValues. (DSP-18763)

6.7.4 DSE core

Changes and enhancements:

• Improved logging identifies which client, keyspace, table, and partition key is rejected when mutation exceeds size threshold. (DB-1051)
• Improved status reporting for nodesync validation list. (DB-2707)
• Enable upgrading and downgrading SSTables using a CQL file that contains DDL statements to recreate the schema. (DB-2951)
• Improved lightweight transactions (LWT) performance. New cassandra.yaml LWT configuration options. (DB-3018)
• Deadlock when replaying schema mutations from commit log during DSE startup. (DB-3190)
• Reject requests from the TPC backpressure queue when they have been on the queue for too long. (DSP-15875)

Resolved issues:

• Tarball installs to create two instances on the same physical server with remote JMX access with binding the separated IPs to port 7199 causes JMX error of Address already in use (Bind failed) because com.sun.management.jmxremote.host is ignored. (DB-2483)
• Slow startup or node hangs when encryption is used. (DB-3050)
• cqlsh EXECUTE AS command does not work. (DB-3098)
• DSE fails to start with ERROR Attempted serializing to buffer exceeded maximum of 65535 bytes. Improved error to identify a workaround for commitlog corruption. (DB-3162)
• AssertionError in temporary buffer pool causes CorruptSSTableException. (DB-3172, DB-3174)
• Memory leak on errors when reading. (DB-3175)
• Bootstrap should fail when the node is not able to fetch the schema from other nodes in the cluster. (DB-3186)
• Increment pending echos when sending gossip echo requests. (DB-3187)
• sstabledowngrade needs write access to the snapshot folder for a different output location. (DB-3231)
• Fixed possible data loss when using DSE Tiered Storage. (DB-3404)
  Warning: If using DSE Tiered Storage, you must immediately upgrade to at least DSE 5.1.16, DSE 6.0.9, or DSE 6.7.4. Be sure to follow the upgrade instructions.

6.7.4 DSE Analytics

Changes and enhancements:

• A warning message is displayed when DSE authentication is enabled, but Spark security is not enabled. (DSP-17273)
• When DSE authentication is enabled, Spark security is forced to be enabled. (DSP-17274)

  dse.yaml	Spark security is enforced
  authentication_options	When enabled: true
  spark_security_enabled	This setting is ignored.
  spark_security_encryption_enabled	This setting is ignored.

Known issues:

• When the Spark security options are not configured in dse.yaml, the native CQL Protocol authentication can be sidestepped with direct access to the Netty RPC client. Although this access should fail to run Spark applications, the CQL authentication can be bypassed on systems with an open Netty port 7077 using Spark RPC. (DSP-17271)
  Solution: Configure the Spark security options in dse.yaml:

  spark_shared_secret_bit_length: 256
                spark_security_enabled: true
                spark_security_encryption_enabled: true

Resolved issues:

• Accessing files from Spark through WebHDFS interface fails with message: java.io.IOException: Content-Length is missing. (DSP-18559)
• BYOS DSEFS access fails with AuthenticationException with dseauth_internal_no_otherschemes. (DSP-18822)
• Submitting many Spark apps will reach the default tombstone_failure_threshold before the default 90 days gc_grace_seconds defined for the system_auth.role_permissions table. (DSP-19098)
  Workaround with this fix:

  1. Manually grant permissions to the user before the user starts Spark jobs:

     GRANT AUTHORIZE, DESCRIBE, MODIFY ON ANY SUBMISSION IN WORKPOOL 'datacenter_name.workpool' TO role_name; 
     

     2. Start Spark jobs for this user.

     3. After all Spark jobs are complete for this user, revoke the permissions for this user.

     REVOKE AUTHORIZE, DESCRIBE, MODIFY ON ANY SUBMISSION IN WORKPOOL 'datacenter_name.workpool' FROM role_name;

6.7.4 DSEFS

Resolved issues:

6.7.4 DSE Graph

Changes and enhancements:

• Configurable memory leak tracking: new nodetool leaksdetection command and Memory leak detection settings options in cassandra.yaml. (DB-3123)
• The graph configuration and gremlin_server sections in ../config/configDseYaml.html#configDseYaml__graph are now correctly commented out at the top level. (DSP-18477)
• gremlin-console startup time is improved. (DSP-11550)

6.7.4 DSE Search

Changes and enhancements:

• Changes to correct uneven distribution of shard requests with the STATIC set cover finder. (DSP-18197)
  • The shard.set.cover.finder default is changed from DYNAMIC to STATIC. After upgrading, you can restore the original behavior with:

    dsetool set_core_property keyspace_name.table_name shard.set.cover.finder=DYNAMIC

  • A new inertia parameter for dsetool set_core_property supports fine tuning. The default value of 1 can be adjusted for environments with vnodes and more than 10 nodes.
• New recommended method for case-insensitive text search, faceting, grouping, and sorting with new LowerCaseStrField custom Solr field type. This type sets field values as lowercase and stores them as lowercase in docValues. (DSP-18763)
  Note: DataStax does not support using the TextField Solr field type with solr.KeywordTokenizer and solr.LowerCaseFilterFactory to achieve single-token, case-insensitive indexing on a CQL text field.

Resolved issues:

• SASI queries don't work on tables with row level access control (RLAC). (DB-3082)
• Documents might not be removed from the index when a key element has value equal to a Solr reserved word. (DSP-17419)
• FQ broken with queryExecutorThreads and timeAllowed set. (DSP-18717)
• Search should error out, rather than timeout, on Solr query with non-existing field list (fl) fields. (DSP-18218)
• While using live indexing, also known as RT or real-time indexing, a race condition can be triggered when concurrently indexing and running heavy facet queries. The race condition fails an assertion that, in turn, fails searcher opening and leaves the index in an inconsistent state. (DSP-18786)

23 April 2019

6.7.3 Components

DSE 6.7.3 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.3 Highlights

High-value benefits of upgrading to DSE 6.7.3 include these highlights:

DSE Database (DSE core) highlights

• Compaction performance improvement with new cassandra.yaml pick_level_on_streaming option. (DB-1658)
• The sstableloader downgrade from DSE to OSS Apache Cassandra is supported with new sstabledowngrade tool. (DB-2756)
  Important: The sstabledowngrade command cannot be used to downgrade system tables or downgrade DSE versions.
• Fix NodeSync failing when validating MV row with empty partition key. (DB-2823)
• nodesync can be enabled and disabled on materialized views (MV). (DB-3008)
• Fixed anti-compaction transaction for atomicity and index building. (DB-3016)
• Remedy deadlock during node startup when calculating disk boundaries. (DB-3028)
• Correct handling of dropped UDT columns in SSTables. (DB-3031)

  Workaround: If issues with UDTs in SSTables exist after upgrade from DSE 6.0.x or DSE 5.0.x, run sstablescrub -e fix-only offline on the SSTables that have or had UDTs that were created in DSE 5.0.x.

• Fix unclosed range tombstones in read response. See Cassandra enhancements for DSE 6.7.3.
• The frame decoding off-heap queue size is configurable and smaller by default. (DB-3047)

DSE Analytics highlights

• AlwaysOn SQL (AOSS) log files, including service.log, are consolidated in system.log. (DSP-18261)
• Support configuration to connect to multiple hosts from BYOS connector with multiple hostnames in DseByosAuthConfFactory. (DSP-18231)
• Fixed a leak in BulkTableWriter. (DSP-18513)

DSE Graph highlights

• Time, date, inet, and duration data types are now supported in graph search indexes. (DSP-17694)
• Some minor DSE GraphFrame code fixes. (DSP-18215)
• Improved usability with simplified vertex and edge loading for single label update. (DSP-18404)
• Operations through gremlin-console run with anonymous permissions. (DSP-18471)

DSE Search highlights

6.7.3 DSE core

Changes and enhancements:

• Compaction performance improvement with new cassandra.yaml pick_level_on_streaming option. (DB-1658)

  Streamed-in SSTables of tables using LCS (leveled compaction strategy) are placed in the same level as the source node, with possible up-leveling. Set to true to save compaction work for operations like nodetool refresh and replacing a node.

• The sstableloader downgrade from DSE to OSS Apache Cassandra is supported with new sstabledowngrade tool. (DB-2756)
• Unused memory in buffer pool. (DB-2788)
• The memory in use in the buffer pool is not identical to the memory allocated. (DB-2904)
• Support for using sstableloader to stream OSS Cassandra 3.x and DSE 5.x data to DSE 6.0 and later. (DB-2909)
• Improved user tools for SSTable upgrades (sstableupgrade) and downgrades (sstabledowngrade). (DB-2950)
• Memory improvements with these supported changes:
  • Configurable memory is supported for offline sstable tools. (DB-2955)

    You can use these environment variables tools:

    • MAX_HEAP_SIZE - defaults to 256 MB
    • MAX_DIRECT_MEMORY - defaults to ((system_memory - heap_size) / 4) with a minimum of 1 GB and a maximum of 8 GB.

    To specify memory on the command line:

    MAX_HEAP_SIZE=2g MAX_DIRECT_MEMORY=10g sstabledowngrade keyspace table

  • Buffer pool, and metrics for the buffer pool, are now in two pools. In cassandra.yaml, file_cache_size_in_mb option sets the file cache (or chunk cache) and new direct_reads_size_in_mb option for all other short-lived read operations. (DB-2958)

    To retrieve the buffer pool metrics:

    nodetool sjk mxdump -q "org.apache.cassandra.metrics:type=CachedReadsBufferPool,name=*"

    nodetool sjk mxdump -q "org.apache.cassandra.metrics:type=DirectReadsBufferPool,name=*"

    For legacy compatibility, org.apache.cassandra.metrics:type=BufferPool still exists and is the same as org.apache.cassandra.metrics:type=CachedReadsBufferPool.

  • cassandra-env.sh respect heap and direct memory values set in jvm.options or as environment variables. (DB-2973)
    The precedence for heap and direct memory is:

    • Environment variables
    • jvm.options
    • calculations in cassandra-env.sh
• AIO is automatically disabled if the chunk cache size is small enough: less or equal to system RAM / 8. (DB-2997)
• nodesync cannot be enabled or disabled on materialized views (MV). (DB-3008)
• Optimized memory usage for direct reads pool when using a high number of LWTs. (DB-3124)

  When not set in cassandra.yaml, the default calculated size of direct_reads_size_in_mb changed from 128 MB to 2 MB per TPC core thread, plus 2 MB shared by non-TPC threads, with a maximum value of 128 MB.

Resolved issues:

• Native server Message.Dispatcher.Flusher task stalls under heavy load. (DB-1814)
• Race in CommitLog can cause failed force-flush-all. (DB-2542)
• Unclosed range tombstones in read response. (DB-2601)
• The sstableloader downgrade from DSE to OSS Apache Cassandra is not supported. New sstabledowngrade tool is required. (DB-2756)
• NodeSync fails when validating MV row with empty partition key. (DB-2823)
• TupleType values with null fields NPE when being made byte-comparable. (DB-2872)
• Nodes in a cluster continue trying to connect to a decommissioned node. (DB-2886)
• Reference leak in SSTableRewriter in sstableupgrade when keepOriginals is true. (DB-2944)
• Hint-dispatcher file-channel not closed, if open() fails with OOM. (DB-2947)
• Hints and metadata should not use buffer pool. (DB-2958)
• Lightweight transactions contention may cause IO thread exhaustion. (DB-2965)
• DIRECT_MEMORY is being calculated using 25% of total system memory if -Xmx is set in jvm.options. (DB-2973)
• Netty direct buffers can potentially double the -XX:MaxDirectMemorySize limit. (DB-2993)
• Increased NIO direct memory because the buffers are not cleaned until GC is run. (DB-2996)
• Unable to upgrade SSTables from DSE 5.0.14 to DSE 6.0.5. (DB-3014)
• Anti-compaction transaction causes temporary data loss. (DB-3016)
• Deadlock during node startup when calculating disk boundaries. (DB-3028)
• Dropped UDT columns in SSTables deserialization are broken after upgrading from DSE 5.0. (DB-3031)
• Limit off-heap frame queues by configurable number of frames and total number of bytes. (DB-3047)
• Mishandling of frozen in complex nested types. (DB-3081)
• cqlsh EXECUTE AS command does not work. (DB-3098)
• 32-bit int overflow in StreamingTombstoneHistogramBuilder during compaction. (DB-3108)
• Too many NotInCacheExceptions (12) in trie index flow. (DB-3120)
• Possible direct memory leak when part of bulk allocation fails. (DB-3125)
• Counters in memtable allocators and buffer pool metrics can be incorrect when out of memory (OOM) failures occur. (DB-3126)
• Memory leak occurs when a read from disk times out. (DB-3127)
• RpcExecutionException does not print the user who is not authorized to perform a certain action. (DSP-15895)
• Leak in BulkTableWriter. (DSP-18513)
• Make the remote host visible in the error message for failed magic number verification. (DSP-18645)

Known issue:

• Possible data loss when using DSE Tiered Storage. (DB-3404)
  Warning: If using DSE Tiered Storage, you must immediately upgrade to at least DSE 5.1.16, DSE 6.0.9, or DSE 6.7.4. Be sure to follow the upgrade instructions.

6.7.3 DSE Analytics

Changes and enhancements:

• Configurable heartbeat detection for AlwaysOn SQL (AOSS). (DSP-16391)
• AlwaysOn SQL (AOSS) log files, including service.log, are consolidated in system.log. (DSP-18261)
• Support configuration to connect to multiple hosts from BYOS connector. (DSP-18231)
• CQL syntax error when single quote is not correctly escaped before including in save cache query to AOSS cache table. (DSP-18418)
• Improved error messaging for AlwaysOn SQL (AOSS) client tool. (DSP-18409)
• dse spark-submit --status <driver_ID> command fails. (DSP-18616)

Resolved issues:

• After client-to-node SSL is enabled, all Spark nodes must also listen on port 7480. (DSP-15744)
• dse client-tool configuration byos-export does not export required Spark properties. (DSP-15938)
• Issue with viewing information for completed jobs when authentication is enabled. (DSP-17854)
• Downloaded Spark JAR files are executable for all users. (DSP-17692)
• Cassandra Spark Connector rejects nested UDT when null. (DSP-17965)
• CassandraHiveMetastore does not unquote predicates for server side filtering. (DSP-18017)
• Spark Cassandra Connector does properly cache manually prepared RegularStatements, see SPARKC-558. (DSP-18075)
• Apache Spark local privilege escalation vulnerability: CVE-2018-11760. (DB-18225)
• Can't access AlwaysOn SQL (AOSS) UI when authorization is enabled. (DSP-18236)
• Invalid options show for dse spark-submit command line help. (DSP-18293)
• Remove class DGFCleanerInterceptor from byos.jar. (DSP-18445)
• GBTClassifier in Spark ML fails when periodic checkpointing is on. (DSP-18450)

Known issue:

• DSE 6.7.3 is not compatible with Zeppelin in SparkR and PySpark 0.8.1. (DSP-18777)

  The Apache Spark™ 2.2.3.4 that is included with DSE 6.7.3 contains the patched protocol and all versions of DSE are compatible with the Scala interpreter.

  However, SparkR and PySpark use only a separate channel for communication with Zeppelin. This protocol was vulnerable to attack from other users on the system and was secured in CVE-2018-11760. Zeppelin in SparkR and PySpark 0.8.1 fails because it does not recognize that Spark 2.2.2 and later contain this patched protocol and attempts to use the old protocol. The Zeppelin patch to recognize this protocol is not available in a released Zeppelin build.

  Solution: Do not upgrade to DSE 6.7.3 if you use SparkR or PySpark. Wait for the Zeppelin release later than 0.8.1 that will recognize that DSE-packaged Spark can use the secured protocol.

6.7.3 DSEFS

Resolved issues:

• Change dsefs:// default port when the DSEFS setting public_port is changed in dse.yaml. (DSP-17962)

  The shortcut dsefs:/// now automatically resolves to broadcastaddress:dsefs.public_port, instead of incorrectly using broadcastaddress:5598 regardless of the configured port.

• weather_sensors demo is updated to use native DSEFS commands instead of dse fs hadoop. (DSP-17708)
• Fix handling of path alternatives in DSEFS shell to provide wildcard support for mkdir and ls commands. (DSP-17768)
  For example, to make several subdirectories with a single command:

  dse fs mkdir -p /datastax/demos/weather_sensors/{byos-daily,byos-monthly,byos-station}

  dse fs mkdir -p {path1,path2}/dir

• Problem with change group ownership of files using the fileSystem.setOwner method. (DSP-18052)

6.7.3 DSE Graph

Changes and enhancements:

• The default for the spark.cassandra.output.ignoreNulls parameter is now true for DSE Graph Frames edge updates. To override this setting, set the spark.cassandra.output.ignoreNulls property to false. (DSP-17377)
• Vertex and especially edge loading is simplified. idColumn function is no longer required. (DSP-18404)

Known issue:

• Improved error reporting for errors during cache-based vertex lookup. AssertionError: Should not happen errors are properly reported depending on the root cause error, for example as a timeout exception. (DSP-18254)
• Potential performance drop involving large table scans with DSE Analytics. DSE Graph OLAP operations such as V().count... and V().groupCount()... may be trivially affected (< 10%). (DSP-18683)

6.7.3 DSE Search

Resolved issues:

• SASI should discard stale static row. (DB-2956)
• Edges are inserted with tombstone values set when inserting a recursive edge with multiple cardinality. When calling g.updateEdges(df), any null entries in the provided data frame should be ignored. (DSP-17377)
• Solr HTTP request for CSV output is blank. The CSVResponseWriter returns only stored fields if a field list is not provided in the URL. (DSP-18029)
  To workaround, specify a field list with the URL:

  /select?q=*%3A*&sort=lst_updt_gdttm+desc&rows=10&fl=field1,field2&wt=csv&indent=true

• Avoid interrupting request threads when an internode handshake fails so that the Lucene file channel lock cannot be interrupted. (DSP-18211)
• Timestamp PK routing on solr_query fails. (DSP-18223)
• Facets and stats queries broken when using queryExecutorThreads. (DSP-18237)

27 February 2019

6.7.2 Components

All components from DSE 6.7.2 are listed.

DSE 6.7.2 is compatible with Apache Cassandra™ 3.11 and includes all production-certified changes from earlier versions.

6.7.2 Resolved issue

6.7.2 Known issue

• Possible data loss when using DSE Tiered Storage. (DB-3404)
  Warning: If using DSE Tiered Storage, you must immediately upgrade to at least DSE 5.1.16, DSE 6.0.9, or DSE 6.7.4. Be sure to follow the upgrade instructions.

11 February 2019

6.7.1 Components

All components from DSE 6.7.1 are listed. Components that are updated for DSE 6.7.1 are indicated.

DSE 6.7.1 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

DSE 6.7.1 Highlights

High-value benefits of upgrading to DSE 6.7.1 include these highlights:

DSE Database (DSE core) highlights

Improvements:

• DSE Metrics Collector aggregates DSE metrics and integrates with existing monitoring solutions to facilitate problem resolution and remediation. (DSP-17319)

Important bug fixes:

• Fixed an issue where heap memory usage seems higher with default file cache settings. (DB-2865)
• Fixed resource leak related to streaming operations that affects tiered storage users. Excessive number of TieredRowWriter threads causing java.lang.OutOfMemoryError. (DB-2463)

DSE Analytics highlights

• You want improved error reporting during Spark job submission. (DSP-16359)
• You are having issues with search improvements for analytics when search is not enabled. (DSP-16465)
• You are moving directories in DSEFS. (DSP-17347)

DSE Graph highlights

DSE Search highlights

• You index timestamp partition keys. (DSP-17761)
• You do a lot of reindexing. (DSP-17975)
• You use frozen maps in a table with DSE Search. (DSP-18073)
• You have non-ASCII characters in your indexed columns. (DSP-17816, DSP-17961)

6.7.1 DSE core

Changes and enhancements:

• The overuse bounds for token allocation is improved when RF = 1 or RF = number of racks. (DB-1552)
• New nodetool rebuild_view command rebuilds materialized views for local data. Existing view data is not cleared. (DB-2451)
• Improved messages for nodetool nodesyncservice ratesimulator command include explanation for single node clusters and when no tables have NodeSync enabled. (DB-2468)
• Improved error message when Netty Epoll library cannot be loaded. (DB-2579)
• New environment variable MAX_DIRECT_MEMORY overrides cassandra.yaml value for how much direct memory (NIO direct buffers) that the JVM can use. (DB-2919)
• New JMX operations for graph MBeans. (DSP-15928)
  • adjacency-cache.size - adjacency cache size attribute
  • adjacency-cache.clear - operation to clean adjacency cache
  • index-cache.size - vertex cache size attribute
  • index-cache.clear - operation to clean vertex cache
  JMX operations are not cluster-aware. Invoke on each node as appropriate to your environment.
• Improved encryption key error reporting. (DSP-17723)
• Changed default values watermark values for DSE Metrics Collector. Configurable with JVM startup properties. (DSP-17733)

Resolved issues:

• Running the nodetool nodesyncservice enable command reports the error NodeSyncRecord constructor assertion failed. (DB-2280)

  Workaround: Before DSE 6.7.1, restart DSE to resolve the issue so that you can execute the command and enable NodeSync without error.

• Read and compaction errors with levelled compaction strategy (LCS). (DB-2446)
• Excessive number of TieredRowWriter threads causing java.lang.OutOfMemoryError (DB-2463)
• The nodetool nodesyncservice ratesimulator -deadline-overrides option is not supported. (DB-2468)
• The nodetool gcstats command output incorrectly reports the GC reclaimed metric in bytes, instead of the expected MB. (DB-2598)
• TypeParser is not thread safe. (DB-2602)
• Possible corruption in compressed files with uncompressed chunks. (DB-2634)
• Incorrect order of application of nodetool garbagecollect leaves tombstones that should be deleted. (DB-2658)
• DSE does not start with Unable to gossip with any peers error if cross_node_timeout is true. (DB-2670)
• Heap and CPU quota exceeded detection for asynchronous JavaScript UDFs is not reliable. (DB-2645)
• Exception should occur when user with no permissions returns no rows on restricted table. (DB-2668)
• User-defined aggregates (UDAs) that instantiate user-defined types (UDTs) break after restart. (DB-2771)
• Memory leak on unfetched continuous paging requests. (DB-2851)
• Batch replay is interrupted and good batches are skipped when a mutation of an unknown table is found. (DB-2855)
• Late continuous paging errors can leave unreleased buffers behind. (DB-2862)
• Heap memory usage is higher with default file cache settings. (DB-2865)
• Prepared statement cache issues when using row-level access control (RLAC) permissions. Existing prepared statements are not correctly invalidated. (DB-2867)
• dsetool does not work when native_transport_interface is set in cassandra.yaml. (DSP-16796)

  To workaround for DSE 6.7.0: Use native_transport_interface_prefer_ipv6 instead.

• Security: java-xmlbuilder is vulnerable to XML external entities (XXE). (DSP-13962)
• Kerberos protocol and QoP parameters are not correctly propagated. (DSP-15455)

Known issue:

• DSE 5.0 SSTables with UDTs will be corrupted after migrating to DSE 5.1, DSE 6.0, and DSE 6.7. (DB-2954, CASSANDRA-15035)
  Important: If the DSE 5.0.x schema contains user-defined types (UDTs), upgrade to at least DSE 5.1.13, DSE 6.0.6, or DSE 6.7.2. The SSTable serialization headers are fixed when DSE is started with the upgraded versions.
• Possible data loss when using DSE Tiered Storage. (DB-3404)
  Warning: If using DSE Tiered Storage, you must immediately upgrade to at least DSE 5.1.16, DSE 6.0.9, or DSE 6.7.4. Be sure to follow the upgrade instructions.

6.7.1 DSE Analytics

6.7.1 DSEFS

Resolved issues:

6.7.1 DSE Graph

6.7.1 DSE Search

5 December 2018

6.7.0 Components

DSE 6.7.0 is compatible with Apache Cassandra™ 3.11 and adds additional production-certified changes.

6.7.0 New features

See DataStax Enterprise 6.7 new features.

6.7.0 DSE database

Known issues:

• DSE 5.0 SSTables with UDTs will be corrupted after migrating to DSE 5.1, DSE 6.0, and DSE 6.7. (DB-2954, CASSANDRA-15035)
  Important: If the DSE 5.0.x schema contains user-defined types (UDTs), upgrade to at least DSE 5.1.13, DSE 6.0.6, or DSE 6.7.2. The SSTable serialization headers are fixed when DSE is started with the upgraded versions.
• The dse.yaml and cassandra.yaml comments incorrectly list PKCS11 as a supported keystore_type. (DB-1629)
  Attention: DSE 6.7.7 added PKCS11 as a supported keystore_type; however, only on nodes with cassandra workloads. The support is specific to DSE 6.7.7 and later releases. For example, PKCS11 is not supported as a keystore_type in DSE 6.7.6 or earlier 6.7.x releases. PKCS11 is not supported in DSE 6.0.x and 5.1.x releases. PKCS11 is not supported as a truststore_type. (DSP-16080)
• Possible data loss when using DSE Tiered Storage. (DB-3404)
  Warning: If using DSE Tiered Storage, you must immediately upgrade to at least DSE 5.1.16, DSE 6.0.9, or DSE 6.7.4. Be sure to follow the upgrade instructions.
• dsetool does not work when native_transport_interface is set in cassandra.yaml. (DSP-16796)

  To workaround: Use native_transport_interface_prefer_ipv6 instead.

6.7.0 DSE Advanced Replication

No updates.

6.7.0 DSE Analytics

6.7.0 DSEFS

6.7.0 DSE Graph

6.7.0 DSE Search

DataStax Studio

• See .

DataStax Bulk Loader

• See DataStax Bulk Loader release notes.