com.datastax.spark.connector.rdd
CassandraJoinRDD
Related Doc:
package rdd
This is slightly different than Scala this.type.
This is slightly different than Scala this.type.
this.type is the unique singleton type of an object which is not compatible with other
instances of the same type, so returning anything other than this is not really possible
without lying to the compiler by explicit casts.
Here SelfType is used to return a copy of the object - a different instance of the same type
Maps each row into object of a different type using provided function taking column value(s) as argument(s).
Maps each row into object of a different type using provided function taking column value(s) as argument(s). Can be used to convert each row to a tuple or a case class object:
sc.cassandraTable("ks", "table") .select("column1") .as((s: String) => s) // yields CassandraRDD[String] sc.cassandraTable("ks", "table") .select("column1", "column2") .as((_: String, _: Long)) // yields CassandraRDD[(String, Long)] case class MyRow(key: String, value: Long) sc.cassandraTable("ks", "table") .select("column1", "column2") .as(MyRow) // yields CassandraRDD[MyRow]
Counts the number of items in this RDD by selecting count(*) on Cassandra table
Counts the number of items in this RDD by selecting count(*) on Cassandra table
This method will create the RowWriter required before the RDD is serialized.
This method will create the RowWriter required before the RDD is serialized. This is called during getPartitions
Adds a CQL ORDER BY clause to the query.
Adds a CQL ORDER BY clause to the query.
It can be applied only in case there are clustering columns and primary key predicate is
pushed down in where.
It is useful when the default direction of ordering rows within a single Cassandra partition
needs to be changed.
When computing a CassandraPartitionKeyRDD the data is selected via single CQL statements from the specified C* Keyspace and Table.
When computing a CassandraPartitionKeyRDD the data is selected via single CQL statements from the specified C* Keyspace and Table. This will be preformed on whatever data is available in the previous RDD in the chain.
Allows to copy this RDD with changing some of the properties
Allows to copy this RDD with changing some of the properties
Adds the limit clause to CQL select statement.
Adds the limit clause to CQL select statement. The limit will be applied for each created Spark partition. In other words, unless the data are fetched from a single Cassandra partition the number of results is unpredictable.
The main purpose of passing limit clause is to fetch top n rows from a single Cassandra partition when the table is designed so that it uses clustering keys and a partition key predicate is passed to the where clause.
Filters currently selected set of columns with a new set of columns
Filters currently selected set of columns with a new set of columns
Adds the PER PARTITION LIMIT clause to CQL select statement.
Adds the PER PARTITION LIMIT clause to CQL select statement. The limit will be applied for every Cassandra Partition. Only Valid For Cassandra 3.6+
RowReaderFactory and ClassTag should be provided from implicit parameters in the constructor of the class implementing this trait
RowReaderFactory and ClassTag should be provided from implicit parameters in the constructor of the class implementing this trait
CassandraTableScanRDD
Narrows down the selected set of columns.
Narrows down the selected set of columns.
Use this for better performance, when you don't need all the columns in the result RDD.
When called multiple times, it selects the subset of the already selected columns, so
after a column was removed by the previous select call, it is not possible to
add it back.
The selected columns are ColumnRef instances. This type allows to specify columns for
straightforward retrieval and to read TTL or write time of regular columns as well. Implicit
conversions included in com.datastax.spark.connector package make it possible to provide
just column names (which is also backward compatible) and optional add .ttl or .writeTime
suffix in order to create an appropriate ColumnRef instance.
Returns the columns to be selected from the table.
Returns the columns to be selected from the table.
Prefetches a batchSize of elements at a time *
Prefetches a batchSize of elements at a time *
Checks for existence of keyspace and table.
Checks for existence of keyspace and table.
Adds a CQL WHERE predicate(s) to the query.
Adds a CQL WHERE predicate(s) to the query.
Useful for leveraging secondary indexes in Cassandra.
Implicitly adds an ALLOW FILTERING clause to the WHERE clause,
however beware that some predicates might be rejected by Cassandra,
particularly in cases when they filter on an unindexed, non-clustering column.
Returns a copy of this Cassandra RDD with specified connector
Returns a copy of this Cassandra RDD with specified connector
Allows to set custom read configuration, e.g.
Allows to set custom read configuration, e.g. consistency level or fetch size.
An RDD that will do a selecting join between
leftRDD and the specified Cassandra Table This will perform individual selects to retrieve the rows from Cassandra and will take advantage of RDDs that have been partitioned with the com.datastax.spark.connector.rdd.partitioner.ReplicaPartitioneritem type on the left side of the join (any RDD)
item type on the right side of the join (fetched from Cassandra)