Spark Cassandra Connector SCC-CHANGES.txt for DSE 5.0.x


A list of DataStax Enterprise 5.0.x production-certified changes for the DataStax Spark Cassandra Connector.
DSE 5.0.15 
* Ignore Solr_query when setting up Writing (SPARKC-541)
* Backport of SPARKC-503 - JWCT Query Throttling (SPARKC-542)
* Allow Split Count to be passed to CassandraSourceRelation (SPARKC-527)
* Allow setting spark.cassandra.concurrent.reads in SparkConf (SPARKC-520)

DSE 5.0.12: 
* SPARKC-318: Nested columns in an MBean cannot be null

* Fix reading null UDTs into POJOs (SPARKC-426)
* Improvements in JWCT Performance and Retry Behavior (SPARKC-507)
* Fix TTL and WRITE time when used with Delete queries (SPARKC-505)
* Error message for WriteTime exception typo corrected

* Fix Shuffling of a Set in Local Node FirstPolicy (SPARKC-496)

DSE 5.0.10: 
* Fix ReplicaPartitioner Endpoint Call (SPARKC-485)
* MultipleRetryPolicy should retry with null (SPARKC-494)

DSE 5.0.9: 
* Protect against overflows in Size Estimates (SPARKC-492)
* Confirm truncation with datasource writes (SPARKC-472)

* Allow Writes to Static Columns and Partition Keys (SPARKC-470)
* SessionProxy Should Proxy all Runtime Interfaces (SPARKC-476)
* Always Use Codec Cache When Reading from Driver Rows (SPARKC-473)

DSE 5.0.8: 
* Optimize Spark Sql Count(*) with CassandraCount (SPARKC-412)
* Fix Incomplete Shading of Guava (SPARKC-461)
* Replace all Logging Dependencies from Spark-Core with Internal Class (SPARKC-460)
* Remove Java8 Dependency in CassandraConnectorConf (SPARKC-462)

DSE 5.0.6: 
 * Find converters for classes on different classloaders (SPARKC-363)
 * Fix CassandraConnector and Session Race Condition (SPARKC-441)
 * Fixed use of connection.local_dc parameter (SPARKC-448) 
 * Add RDD.deleteFromCassandra() method (SPARKC-349) (backport)
 * Left join with Cassandra (SPARKC-181) (backport)
 * CustomDriverConverter support for DataFrame Conversions (SPARKC-440)

DSE 5.0.5: 
 * Added SSL client authentication (SPARKC-359)
 * Correct order of "Limit" and "orderBy" clauses in JWCT (SPARKC-433)
 * Improved error messages on CreateCassandraTable from DataFrame (SPARKC-428)
 * Improved backwards compatibility with older Cassandra versions
   (SPARKC-387, SPARKC-402)
 * Fix partitioner on Integer.MIN_VALUE (SPARKC-419)

DSE 5.0.4: 
 * Fixed shading in artifacts published to Maven

 * Disallow TimeUUID Predicate Pushdown (SPARKC-405)
 * Avoid overflow on SplitSizeInMB param (SPARKC-413)
 * Fix conversion of LocalDate to Joda LocalDate (SPARKC-391)
 * Shade Guava to avoid conflict with outdated Guava in Spark (SPARKC-355)

DSE 5.0.0: 
 * SparkSql write supports TTL per row (SPARKC-345)
 * Make Repartition by Cassandra Replica Deterministic (SPARKC-278)
 * Improved performance by caching converters and Java driver codecs
 * Added support for driver.LocalDate (SPARKC-385)
 * Accept predicates with indexed partition columns (SPARKC-348)
 * Retry schema checks to avoid Java Driver debouncing (SPARKC-379)
 * Fix compatibility with Cassandra 2.1.X with Single Partitions/In queries
 * Use executeAsync while joining with C* table (SPARKC-233)
 * Fix support for C* Tuples in Dataframes (SPARKC-357)

1.6.0 M2
 * Performance improvement: keyBy creates an RDD with CassandraPartitioner
   so shuffling can be avoided in many cases, e.g. when keyBy is followed
   by groupByKey or join (SPARKC-330)
 * Improved exception message when data frame is to be saved in
   non-empty Cassandra table (SPARKC-338)
 * Support for Joda time for Cassandra date type (SPARKC-342)
 * Don't double resolve the paths for port locks in embedded C*
   (contribution by crakjie)
 * Drop indices that cannot be used in predicate pushdown (SPARKC-347)
 * Added support for IF NOT EXISTS (SPARKC-362)
 * Nested Optional Case Class can be save as UDT (SPARKC-346)
 * Merged Java API into main module (SPARKC-335)
 * Upgraded to Spark 1.6.1 (SPARKC-344)
 * Fix NoSuchElementException when fetching database schema from Cassandra
 * Removed the ability to specify cluster alias directly and added some helper methods
   which make it easier to configure Cassandra related data frames (SPARKC-289)

1.6.0 M1
 * Adds the ability to add additional Predicate Pushdown Rules at Runtime (SPARKC-308)
 * Added CassandraOption for Skipping Columns when Writing to C* (SPARKC-283)
 * Upgrade Spark to 1.6.0 and add Apache Snapshot repository to resolvers (SPARKC-272, SPARKC-298, SPARKC-305)
 * Includes all patches up to 1.5.0.


* Find converters for classes on different classloaders (SPARKC-363)
* Disallow TimeUUID predicate pushdown (SPARKC-405)
* Avoid overflow on SplitSizeInMB param (SPARKC-413)
* Fix CassandraConnector and Session Race Condition (SPARKC-441)
* Correct Order of "limit" and "orderBy" clauses in JWCT (SPARKC-443)
* Fixed use of connection.local_dc parameter (SPARKC-448)

 * Includes all patches up to 1.4.4

 * Fixed assembly build (SPARKC-311)
 * Upgrade Cassandra version to 3.0.2 by default and allow to specify arbitrary Cassandra version for
   integration tests through the command line (SPARKC-307)
 * Upgrade Cassandra driver to 3.0.0 GA
 * Includes all patches up to 1.4.2.

1.5.0 RC1
 * Fix special case types in SqlRowWriter (SPARKC-306)
 * Fix sbt assembly
 * Create Cassandra Schema from DataFrame (SPARKC-231)
 * JWCT inherits Spark Conf from Spark Context (SPARKC-294)
 * Support of new Cassandra Date and Time types (SPARKC-277)
 * Upgrade Cassandra driver to 3.0.0-rc1

1.5.0 M3
 * Added ColumRef child class to represent functions calls (SPARKC-280)
 * Warn if Keep_alive_ms is less than spark batch size in streaming (SPARKC-228)
 * Fixed real tests (SPARKC-247)
 * Added support for tinyint and smallint types (SPARKC-269)
 * Updated Java driver version to 3.0.0-alpha4; Codec API changes (SPARKC-285)
 * Updated Java driver version to 3.0.0-alpha3 (SPARKC-270)
 * Changed the way CassandraConnectorSource is obtained due to SPARK-7171 (SPARKC-268)
 * Change write ConsistencyLevel to LOCAL_QUORUM (SPARKC-262)
 * Parallelize integration tests (SPARKC-293)
 * Includes all patches up to 1.4.1.

1.5.0 M2
 * Bump Java Driver to 2.2.0-rc3, Guava to 16.0.1 and test against Cassandra 2.2.1 (SPARKC-229)
 * Includes all patches up to 1.4.0.

1.5.0 M1
 * Added ability to build against unreleased Spark versions (SPARKC-242)
 * Spark 1.5 initial integration (SPARKC-241)