Querying Cassandra data using Spark SQL in Scala

You can execute Spark SQL queries in Scala by starting the Spark shell. When you start Spark, DataStax Enterprise sets the context to allow you to run Spark SQL queries against Cassandra tables.

When you start Spark, DataStax Enterprise sets the context to allow you to run Spark SQL queries against Cassandra tables. The context object is named sqlContext and is an instance of HiveContext. HiveContext is a superset of SqlContext and uses the Hive metastore. Use the setKeyspace method to connect to a Cassandra keyspace, and then use the sql method to execute the query.

Procedure

  1. Start the Spark shell. 
    dse spark
  2. Use the sql method to pass in the query, storing the result in a variable. 
    val results = sqlContext.sql("SELECT * from my_keyspace_name.my_table")
  3. Use the returned data. 
    results.collect().foreach(println)
    CassandraRow{type_id: 1, value: 9685.807}
CassandraRow{type_id: 2, value: -9775.808}