Class/Object

com.datastax.bdp.graph.spark.sql.vertex

SingleLabelVertexSourceRelation

Related Docs: object SingleLabelVertexSourceRelation | package vertex

Permalink

class SingleLabelVertexSourceRelation extends BaseRelation with InsertableRelation with PrunedFilteredScan with Logging with Serializable

VertexSourceRelation return DF for one vertex label only It returns DF with all graph vertex properties by default. It is done to simplify UNION of different vertexes. Unavailable vertex properties are returned as null. spark cassandra connector mapping is used to map C* type to DF types. mutli vertices are returned as array of values: ArrayType[DataType] if property definition has metadata, following struct will be returned StructType[value: FieldType, metaFields*]

multi-meta property will be: ArrayType[StructType[value: FieldType, meta...]]

Example of the schema: |member_id|community_id|skill| name|startTime|location|endTime|since| for one vertex it will return |member_id|community_id|NULL | name|NULL |location|NULL |NULL | if user defines columns with select("skill", "name", "startTime", "location") call, C* source will be asked for: |~~property-key-id| all vertex meta properties|idcolumns*|name|location| the source will return all requested properties (with meta struct if defined by schema). |NULL|name|null|location

Linear Supertypes
Serializable, Serializable, Logging, PrunedFilteredScan, InsertableRelation, BaseRelation, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SingleLabelVertexSourceRelation
  2. Serializable
  3. Serializable
  4. Logging
  5. PrunedFilteredScan
  6. InsertableRelation
  7. BaseRelation
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SingleLabelVertexSourceRelation(label: String, tableRef: TableRef, cassandraSourceRelation: BaseRelation)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def buildC2VertexMaping(resultSchema: StructType): (Seq[(Int, CassandraRowConverter)], Map[Int, CassandraRowConverter], Map[Int, Int], Seq[String], Int)

    Permalink

    build C* request, and id mapping according to following example.

    build C* request, and id mapping according to following example.

    Example of the schema: val schema: |member_id|community_id|skill| name|startTime|location|endTime|since| for one vertex it will return |member_id|community_id|NULL | name|NULL |location|NULL |NULL | val resultSchema: ("skill", "name", "startTime", "location"). C* source will be: val withIdAndMeta: |~~property-key-id| all vertex meta properties|idcolumns*|name|location| the source will return all requested properties (with meta struct if defined by schema). |NULL|name|null|location

    Attributes
    protected
  6. def buildScan(requiredColumns: Array[String], filters: Array[Filter]): RDD[Row]

    Permalink

    call cassandraSourceRelation and then aggregate properties in the single row C* returns sorted rows, so we use it in mapPartition functions when yield new rows.

    call cassandraSourceRelation and then aggregate properties in the single row C* returns sorted rows, so we use it in mapPartition functions when yield new rows. it is a kind of copy of SpanByKey, but without object creation overhead. the request will contains |-~~property_key_id-|-all metaData columns-|-vertex id columns-|-filteredRequiredColumns-|

    filteredRequiredColumns = requiredColumns intersect cassandraColumns to prevent no such column failure.

    It will return |-requiredColumns-|

    filters

    can not be pushed down to cassandra dirreclty, they are not used yet.

    returns

    |-requiredColumns-| RDD

    Definition Classes
    SingleLabelVertexSourceRelation → PrunedFilteredScan
  7. val cassandraColumns: Seq[String]

    Permalink
  8. val cassandraConnector: CassandraConnector

    Permalink
  9. val cassandraSourceRelation: BaseRelation

    Permalink
  10. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def delete(idDF: DataFrame, properties: String*): Unit

    Permalink

    all id columns should be in place.

    all id columns should be in place. the method will unaggregate properties to the C* table view.* and call rdd.deleteFromCassandra()

    idDF

    data frame with all vertex id columns. It is ("community_id", "member_id") for standart id

    properties

    , optional, only specified properties will be deleted, vertex will be deleted if empty

  12. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  14. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  16. val graphSchema: SerializableSchema

    Permalink
  17. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  18. val idColumns: Buffer[String]

    Permalink
  19. def insert(updateData: DataFrame, overwrite: Boolean): Unit

    Permalink

    all id columns should be in place.

    all id columns should be in place. the method will unaggregate properties to the C* table view.*

    Definition Classes
    SingleLabelVertexSourceRelation → InsertableRelation
  20. def isEmpty(): Boolean

    Permalink

    check that table is empty 'select * from table limit 1' is the fastest way

  21. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  22. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  23. val label: String

    Permalink
  24. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  25. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  26. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  27. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  28. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  29. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  30. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  31. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  32. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  33. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  34. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  35. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  36. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  37. val needConversion: Boolean

    Permalink

    allows us to use InternalRows instead of public API rows.

    allows us to use InternalRows instead of public API rows.

    Definition Classes
    SingleLabelVertexSourceRelation → BaseRelation
  38. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  39. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  40. val propTTLSeconds: Map[Integer, Integer]

    Permalink

    Maps property key ID to that key's effective TTL on this vertex label.

    Maps property key ID to that key's effective TTL on this vertex label. The "effective TTL" is the minimum of this vertex label's TTL and the property key's TTL. If neither the property key nor the vertex label has a TTL, then the corresponding property key ID will be absent from this map.

  41. val readConf: ReadConf

    Permalink
  42. def schema: StructType

    Permalink
    Definition Classes
    SingleLabelVertexSourceRelation → BaseRelation
  43. lazy val schemaMap: Map[String, StructField]

    Permalink
  44. def sizeInBytes: Long

    Permalink
    Definition Classes
    BaseRelation
  45. def sparkConf: SparkConf

    Permalink
  46. def sqlContext: SQLContext

    Permalink
    Definition Classes
    SingleLabelVertexSourceRelation → BaseRelation
  47. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  48. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  49. def unhandledFilters(filters: Array[Filter]): Array[Filter]

    Permalink
    Definition Classes
    BaseRelation
  50. val vertex: Vertex

    Permalink
  51. val vertexTTLSeconds: Int

    Permalink

    The TTL of this vertex label, in seconds.

    The TTL of this vertex label, in seconds. Zero means none is set.

  52. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  53. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  54. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  55. val writeConf: WriteConf

    Permalink

Inherited from Serializable

Inherited from Serializable

Inherited from Logging

Inherited from PrunedFilteredScan

Inherited from InsertableRelation

Inherited from BaseRelation

Inherited from AnyRef

Inherited from Any

Ungrouped