Spark Cassandra Connector SCC-CHANGES.txt for DSE 5.0.x
SCC-CHANGES.txt
A list of DataStax Enterprise 5.0.x production-certified changes for the DataStax Spark
Cassandra Connector.
DSE 5.0.15
1.6.12
* Ignore Solr_query when setting up Writing (SPARKC-541)
* Backport of SPARKC-503 - JWCT Query Throttling (SPARKC-542)
* Allow Split Count to be passed to CassandraSourceRelation (SPARKC-527)
* Allow setting spark.cassandra.concurrent.reads in SparkConf (SPARKC-520)
DSE 5.0.12:
1.6.11
* SPARKC-318: Nested columns in an MBean cannot be null
1.6.10
* Fix reading null UDTs into POJOs (SPARKC-426)
* Improvements in JWCT Performance and Retry Behavior (SPARKC-507)
* Fix TTL and WRITE time when used with Delete queries (SPARKC-505)
* Error message for WriteTime exception typo corrected
1.6.9
* Fix Shuffling of a Set in Local Node FirstPolicy (SPARKC-496)
DSE 5.0.10:
1.6.8
* Fix ReplicaPartitioner Endpoint Call (SPARKC-485)
* MultipleRetryPolicy should retry with null (SPARKC-494)
DSE 5.0.9:
1.6.7
* Protect against overflows in Size Estimates (SPARKC-492)
* Confirm truncation with datasource writes (SPARKC-472)
1.6.6
* Allow Writes to Static Columns and Partition Keys (SPARKC-470)
* SessionProxy Should Proxy all Runtime Interfaces (SPARKC-476)
* Always Use Codec Cache When Reading from Driver Rows (SPARKC-473)
DSE 5.0.8:
1.6.5
* Optimize Spark Sql Count(*) with CassandraCount (SPARKC-412)
* Fix Incomplete Shading of Guava (SPARKC-461)
* Replace all Logging Dependencies from Spark-Core with Internal Class (SPARKC-460)
* Remove Java8 Dependency in CassandraConnectorConf (SPARKC-462)
DSE 5.0.6:
1.6.4
* Find converters for classes on different classloaders (SPARKC-363)
* Fix CassandraConnector and Session Race Condition (SPARKC-441)
* Fixed use of connection.local_dc parameter (SPARKC-448)
* Add RDD.deleteFromCassandra() method (SPARKC-349) (backport)
* Left join with Cassandra (SPARKC-181) (backport)
* CustomDriverConverter support for DataFrame Conversions (SPARKC-440)
DSE 5.0.5:
1.6.3
* Added SSL client authentication (SPARKC-359)
* Correct order of "Limit" and "orderBy" clauses in JWCT (SPARKC-433)
* Improved error messages on CreateCassandraTable from DataFrame (SPARKC-428)
* Improved backwards compatibility with older Cassandra versions
(SPARKC-387, SPARKC-402)
* Fix partitioner on Integer.MIN_VALUE (SPARKC-419)
DSE 5.0.4:
1.6.2
* Fixed shading in artifacts published to Maven
1.6.1
* Disallow TimeUUID Predicate Pushdown (SPARKC-405)
* Avoid overflow on SplitSizeInMB param (SPARKC-413)
* Fix conversion of LocalDate to Joda LocalDate (SPARKC-391)
* Shade Guava to avoid conflict with outdated Guava in Spark (SPARKC-355)
DSE 5.0.0:
1.6.0
* SparkSql write supports TTL per row (SPARKC-345)
* Make Repartition by Cassandra Replica Deterministic (SPARKC-278)
* Improved performance by caching converters and Java driver codecs
(SPARKC-383)
* Added support for driver.LocalDate (SPARKC-385)
* Accept predicates with indexed partition columns (SPARKC-348)
* Retry schema checks to avoid Java Driver debouncing (SPARKC-379)
* Fix compatibility with Cassandra 2.1.X with Single Partitions/In queries
(SPARKC-376)
* Use executeAsync while joining with C* table (SPARKC-233)
* Fix support for C* Tuples in Dataframes (SPARKC-357)
1.6.0 M2
* Performance improvement: keyBy creates an RDD with CassandraPartitioner
so shuffling can be avoided in many cases, e.g. when keyBy is followed
by groupByKey or join (SPARKC-330)
* Improved exception message when data frame is to be saved in
non-empty Cassandra table (SPARKC-338)
* Support for Joda time for Cassandra date type (SPARKC-342)
* Don't double resolve the paths for port locks in embedded C*
(contribution by crakjie)
* Drop indices that cannot be used in predicate pushdown (SPARKC-347)
* Added support for IF NOT EXISTS (SPARKC-362)
* Nested Optional Case Class can be save as UDT (SPARKC-346)
* Merged Java API into main module (SPARKC-335)
* Upgraded to Spark 1.6.1 (SPARKC-344)
* Fix NoSuchElementException when fetching database schema from Cassandra
(SPARKC-341)
* Removed the ability to specify cluster alias directly and added some helper methods
which make it easier to configure Cassandra related data frames (SPARKC-289)
1.6.0 M1
* Adds the ability to add additional Predicate Pushdown Rules at Runtime (SPARKC-308)
* Added CassandraOption for Skipping Columns when Writing to C* (SPARKC-283)
* Upgrade Spark to 1.6.0 and add Apache Snapshot repository to resolvers (SPARKC-272, SPARKC-298, SPARKC-305)
* Includes all patches up to 1.5.0.
********************************************************************************
1.5.2
* Find converters for classes on different classloaders (SPARKC-363)
* Disallow TimeUUID predicate pushdown (SPARKC-405)
* Avoid overflow on SplitSizeInMB param (SPARKC-413)
* Fix CassandraConnector and Session Race Condition (SPARKC-441)
* Correct Order of "limit" and "orderBy" clauses in JWCT (SPARKC-443)
* Fixed use of connection.local_dc parameter (SPARKC-448)
1.5.1
* Includes all patches up to 1.4.4
1.5.0
* Fixed assembly build (SPARKC-311)
* Upgrade Cassandra version to 3.0.2 by default and allow to specify arbitrary Cassandra version for
integration tests through the command line (SPARKC-307)
* Upgrade Cassandra driver to 3.0.0 GA
* Includes all patches up to 1.4.2.
1.5.0 RC1
* Fix special case types in SqlRowWriter (SPARKC-306)
* Fix sbt assembly
* Create Cassandra Schema from DataFrame (SPARKC-231)
* JWCT inherits Spark Conf from Spark Context (SPARKC-294)
* Support of new Cassandra Date and Time types (SPARKC-277)
* Upgrade Cassandra driver to 3.0.0-rc1
1.5.0 M3
* Added ColumRef child class to represent functions calls (SPARKC-280)
* Warn if Keep_alive_ms is less than spark batch size in streaming (SPARKC-228)
* Fixed real tests (SPARKC-247)
* Added support for tinyint and smallint types (SPARKC-269)
* Updated Java driver version to 3.0.0-alpha4; Codec API changes (SPARKC-285)
* Updated Java driver version to 3.0.0-alpha3 (SPARKC-270)
* Changed the way CassandraConnectorSource is obtained due to SPARK-7171 (SPARKC-268)
* Change write ConsistencyLevel to LOCAL_QUORUM (SPARKC-262)
* Parallelize integration tests (SPARKC-293)
* Includes all patches up to 1.4.1.
1.5.0 M2
* Bump Java Driver to 2.2.0-rc3, Guava to 16.0.1 and test against Cassandra 2.2.1 (SPARKC-229)
* Includes all patches up to 1.4.0.
1.5.0 M1
* Added ability to build against unreleased Spark versions (SPARKC-242)
* Spark 1.5 initial integration (SPARKC-241)
********************************************************************************