Accessing Cassandra from Spark

DataStax Enterprise integrates Spark with Cassandra. Cassandra tables are fully usable from Spark without Shark.

DataStax Enterprise integrates Spark with Cassandra. Cassandra tables are fully usable from Spark without Shark. To access Cassandra from a Spark application, follow instructions in the Spark example Portfolio Manager demo using Spark. To access Cassandra from the Spark Shell, just run the dse spark command and follow instructions in subsequent sections.

$ dse spark

Welcome to
         ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 0.9.1
      /_/

Using Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_25)
Type in expressions to have them evaluated.
Type :help for more information.
Creating SparkContext...
2014-06-26 22:52:05.295 java[94799:1703] Unable to load realm info from SCDynamicStore
Created spark context..
Spark context available as sc.
Type in expressions to have them evaluated.
Type :help for more information.

scala>

DataStax Enterprise uses the Spark Cassandra Connector to provide Cassandra integration for Spark. By running the Spark Shell in DataStax Enterprise, you have access to the enriched Spark Context object (sc) for accessing Cassandra directly. DataStax Enterprise also provides additional functionality through the Cassandra Context object (cc). Using the Cassandra Context object, you can easily access Cassandra by using keyspace/table autocompletion and automatically generated case classes for all the user tables. Follow instructions in subsequent sections to use these features.