Class/Object

com.datastax.bdp.graph.spark.graphframe.classic

ClassicDseGraphFrame

Related Docs: object ClassicDseGraphFrame | package classic

Permalink

class ClassicDseGraphFrame extends DseGraphFrame

Linear Supertypes
DseGraphFrame, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ClassicDseGraphFrame
  2. DseGraphFrame
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ClassicDseGraphFrame(gf: GraphFrame, name: Option[String] = None, schema: Option[SerializableSchema] = None, graphOptions: Map[String, String] = Map.empty)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def E(edgesIds: AnyRef*): DseGraphTraversal[Edge]

    Permalink

    Return graph traversal that supports a subset of TinkerPop3 traversal steps

    Return graph traversal that supports a subset of TinkerPop3 traversal steps

    edgesIds

    to start traverse with

    returns

    GraphTraversal[Edge] for the filtered graph

    Definition Classes
    DseGraphFrame
    Annotations
    @varargs()
  5. def E: DseGraphTraversal[Edge]

    Permalink

    Return graph traversal that supports a subset of TinkerPop3 traversal steps

    Return graph traversal that supports a subset of TinkerPop3 traversal steps

    returns

    GraphTraversal[Edge] for the graph

    Definition Classes
    DseGraphFrame
  6. def V(vertexIds: AnyRef*): DseGraphTraversal[Vertex]

    Permalink

    Return graph traversal that supports subset of TinkerPop3 traversal steps

    Return graph traversal that supports subset of TinkerPop3 traversal steps

    vertexIds

    to start traverse with

    returns

    GraphTraversal[Vertex] for the filtered graph

    Definition Classes
    DseGraphFrame
    Annotations
    @varargs()
  7. def V: DseGraphTraversal[Vertex]

    Permalink

    Return graph traversal that supports subset of TinkerPop3 traversal steps

    Return graph traversal that supports subset of TinkerPop3 traversal steps

    returns

    GraphTraversal[Vertex] for the graph

    Definition Classes
    DseGraphFrame
  8. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  9. def cache(): ClassicDseGraphFrame.this.type

    Permalink

    proxy call to gf.cache()

    proxy call to gf.cache()

    returns

    this

    Definition Classes
    DseGraphFrame
  10. def cleanUp: String

    Permalink

    Remove any invalid vertex property and edge entries from the database backend.

    Remove any invalid vertex property and edge entries from the database backend. Call this method if you get internal errors or inconsistent results from any graph queries it is strongly recommended to run nodetool repair graphName before and then again after this call the call revises graph database storage and fixes following problems

    • delete vertex properties entries of non-existent vertex
    • delete vertex properties with unknown ids
    • delete edges with unknown/removed edge or vertex labels
    • delete edges that points to non-existent vertices
    • restore second copy of the edge
    • delete second copy of the edge if primary record exist
    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  11. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  12. val clusterName: String

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  13. lazy val connector: CassandraConnector

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  14. val connectorOptions: Map[String, String]

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  15. def deleteEdgeProperties(df: DataFrame, properties: String*): Unit

    Permalink

    clean edges properties

    clean edges properties

    properties

    delete only selected properties not entire row

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
    Annotations
    @varargs()
  16. def deleteEdges(df: DataFrame, cache: Boolean = true): Unit

    Permalink

    delete graph edges.

    delete graph edges. 4 id columns should be passed to the method

    +--------------------+--------------------+-------+--------------------+
    |                 src|                 dst| ~label|                  id|
    +--------------------+--------------------+-------+--------------------+
    |god:THxdAAAAAAAAAAAA|titan:J474AAAAAAA...| father|da0a9900-8fe1-11e...|
    +--------------------+--------------------+-------+--------------------+
    df

    data frame with edge ids: src,dst,~label, id

    cache

    cache df before processing, true by default for consistence updates. two C* entries need to be deleted for one edge, so no reloads expected between this two calls.

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  17. def deleteEdges(df: DataFrame): Unit

    Permalink

    shortcut for deleteEdges(df: DataFrame, cache: Boolean = true) for Java

    shortcut for deleteEdges(df: DataFrame, cache: Boolean = true) for Java

    Definition Classes
    DseGraphFrame
  18. def deleteVertexProperties(df: DataFrame, properties: Seq[String], labels: Seq[String] = Seq.empty, cache: Boolean = true): Unit

    Permalink

    clean vertex properties with meta properties

    clean vertex properties with meta properties

    properties

    property names to delete

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  19. def deleteVertexProperties(df: DataFrame, properties: String*): Unit

    Permalink

    clean vertex properties with meta properties

    clean vertex properties with meta properties

    properties

    property names to delete

    Definition Classes
    DseGraphFrame
    Annotations
    @varargs()
  20. def deleteVertices(label: String): Unit

    Permalink

    delete all vertices with given label

    delete all vertices with given label

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  21. def deleteVertices(df: DataFrame, labels: Seq[String] = Seq.empty, cache: Boolean = true): Unit

    Permalink

    delete vertices and all related edges

    delete vertices and all related edges

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  22. def dropIsolatedVertices(): DseGraphFrame

    Permalink

    proxy call to gf.dropIsolatedVertices()

    proxy call to gf.dropIsolatedVertices()

    returns

    new filtered DseGraphFrame

    Definition Classes
    DseGraphFrame
  23. var dseGraphName: Option[String]

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  24. lazy val edgeIdColumnNames: Seq[String]

    Permalink
    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  25. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  26. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  27. def filterEdges(conditionExpr: String): DseGraphFrame

    Permalink

    proxy call to gf.filterEdges()

    proxy call to gf.filterEdges()

    returns

    new filtered DseGraphFrame

    Definition Classes
    DseGraphFrame
  28. def filterEdges(condition: Column): DseGraphFrame

    Permalink

    proxy call to gf.filterEdges()

    proxy call to gf.filterEdges()

    returns

    new filtered DseGraphFrame

    Definition Classes
    DseGraphFrame
  29. def filterVertices(conditionExpr: String): DseGraphFrame

    Permalink

    proxy call to gf.filterVertices()

    proxy call to gf.filterVertices()

    returns

    new filtered DseGraphFrame

    Definition Classes
    DseGraphFrame
  30. def filterVertices(condition: Column): DseGraphFrame

    Permalink

    proxy call to gf.filterVertices()

    proxy call to gf.filterVertices()

    returns

    new filtered DseGraphFrame

    Definition Classes
    DseGraphFrame
  31. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  32. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  33. val gf: GraphFrame

    Permalink
    Definition Classes
    DseGraphFrame
  34. def graphName: String

    Permalink

    Returns the graph name of this DseGraphFrame.

    Returns the graph name of this DseGraphFrame.

    Definition Classes
    DseGraphFrame
    Exceptions thrown

    NoSuchElementException if the graph name is not set.

  35. val graphOptions: Map[String, String]

    Permalink
    Definition Classes
    DseGraphFrame
  36. def graphSchema: SerializableSchema

    Permalink

    Return schema of this graph based on the name NoSuchElementException will be thrown if the graph name is unknown and schema can not be retrieved

    Return schema of this graph based on the name NoSuchElementException will be thrown if the graph name is unknown and schema can not be retrieved

    returns

    Graph Schema

  37. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  38. def idColumn(labelColumn: Column, idColumns: Column*): Column

    Permalink

    Utility method to generate GraphFrame compatible ids, if a mixed set of labels is in the DF.

    Utility method to generate GraphFrame compatible ids, if a mixed set of labels is in the DF. It is slower than idColumn(label: String, idColumns: Column*): Column The id is added automatically when vertex is inserted, if inserted columns has the same names as in graph schema It is not possible for edges as you need to point both src and dst ids. Usage:

    val updateEdgeDF = sourceDF.select(
      gf.idColumn(col("srcLabel"), col("srcId")) as "src",
      gf.idColumn(col("dstLabel"), col("dstId")) as "dst",
      col("label") as "~label",
      gf.randomEdgeIdColumn,
      col("property"))
    
    gf.updateEdges(updateEdgeDF)

    If different labels have different id format use case statement to sort them:

    when(col("srcLabel") === "1format", col("src1Id")).when(col("srcLabel") === "2format", col("src2Id")).otherwise(col("src3Id")) as "src"
    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
    Annotations
    @varargs()
  39. def idColumn(label: String, idColumns: Column*): Column

    Permalink

    Utility method to generate GraphFrame compatible ids.

    Utility method to generate GraphFrame compatible ids. The id is added automatically when vertex is inserted, if inserted columns has the same names as in graph schema It is not possible for edges as you need to point both src and dst ids. Usage:

    val updateEdgeDF = sourceDF.select(
      gf.idColumn("srcLabel", col("srcId")) as "src",
      gf.idColumn("dstLabel", col("dstId")) as "dst",
      col("label") as "~label",
      gf.randomEdgeIdColumn,
      col("property"))
    
    gf.updateEdges(updateEdgeDF)
    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
    Annotations
    @varargs()
  40. def io(url: String): DseGraphTraversal[Vertex]

    Permalink

    Performs a read or write based operation on the Graph backing this GraphTraversalSource.

    Performs a read or write based operation on the Graph backing this GraphTraversalSource. This step can be accompanied by the Object) modulator for further configuration and must be accompanied by a GraphTraversal#read() or GraphTraversal#write() modulator step which will terminate the traversal.

    url

    the url of file in distributed file system or JDBC connection or the name of file in default file system for which the read or write will apply - note that the context of how this parameter is used is wholly dependent on the implementation. i.e cassandra read/writer implementation will ignore this path and read table name from parameters.

    returns

    the traversal with the { @link IoStep} added

    Definition Classes
    DseGraphFrame
  41. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  42. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  43. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  44. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  45. def persist(storageLevel: StorageLevel): ClassicDseGraphFrame.this.type

    Permalink

    proxy call to gf.persist()

    proxy call to gf.persist()

    returns

    this

    Definition Classes
    DseGraphFrame
  46. def persist(): ClassicDseGraphFrame.this.type

    Permalink

    proxy call to gf.persist()

    proxy call to gf.persist()

    returns

    this

    Definition Classes
    DseGraphFrame
  47. lazy val spark: SparkSession

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  48. lazy val sqlContext: SQLContext

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame
  49. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  50. def toExternalEdgeId(label: String, srcId: String, dstId: String, ids: Seq[Any], schema: StructType): AnyRef

    Permalink

    Example output:

    Example output:

    {~label=edge,
    ~out_vertex={~label=vertex, community_id=1888030080, member_id=0},
    ~in_vertex={~label=custom, name=Name, array=1},
    ~local_id=2f3671c0-96d0-11e6-9882-f74edf21f349}
    label

    Edge label

    srcId

    Source vertex id

    dstId

    Destination vertex id

    ids

    Edge ids

    schema

    Associated DataFrame schema

    returns

    External ID object

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  51. def toExternalVertexId(id: String): AnyRef

    Permalink

    id

    String of vertex ID

    returns

    External ID object

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  52. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  53. def toSyntheticVertexId(id: AnyRef): String

    Permalink
  54. def unpersist(blocking: Boolean): ClassicDseGraphFrame.this.type

    Permalink

    proxy call to gf.unpersist()

    proxy call to gf.unpersist()

    returns

    this

    Definition Classes
    DseGraphFrame
  55. def unpersist(): ClassicDseGraphFrame.this.type

    Permalink

    proxy call to gf.unpersist()

    proxy call to gf.unpersist()

    returns

    this

    Definition Classes
    DseGraphFrame
  56. def updateEdges(outVertexLabel: String, edgeLabel: String, inVertexLabel: String, df: DataFrame): Unit

    Permalink

    update or insert edges.

    update or insert edges. this method accept natural vertex id columns. Classic graph out vertex id column names should start with "outVertexLabel_" prefix and in names with "inVertexLabel_". Core graph uses DSE-DB edge table schema. The minimal df schema is: 2 id and 0 or more property columns

    For example, suppose we have the following vertex and edge label definition (Core Graph)

    schema.vertexLabel("person")
      .partitionKey("name", "ssn")
      .clusteringKey("age")
      .properties("address", "coffeePerDay")
      .create()
    
    schema.vertexLabel("software")
      .partitionKey("name")
      .clusteringKey("version", "lang")
      .properties("temp", "static_property")
      .create()
    
    schema.edgeLabel("created")
      .multiple()
      .properties("weight")
      .create()
    
    schema.edgeLabel("created")
      .connection("person", "software")
      .add()

    Edge updates can be carried out like this

    scala> g.updateEdges("person", "created", "software", createdDF)

    where the dataframe has the following column names, note they follow the vertexLabel_idColName naming convention

    scala> createdDF.show
    +-----------+-----------+----------+-------------+----------------+-------------+------+
    |person_name| person_ssn|person_age|software_name|software_version|software_lang|weight|
    +-----------+-----------+----------+-------------+----------------+-------------+------+
    |      rocco|111-11-1111|        21|         chat|             1.0|        scala|   2.0|
    +-----------+-----------+----------+-------------+----------------+-------------+------+

    Note: The dataframe is not cached by this function. The dataframe should be persisted by the user if a dynamic data source is used.

    df

    data frame with edge ids and update columns

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  57. def updateEdges(df: DataFrame, cache: Boolean = true): Unit

    Permalink

    update this graph edges.

    update this graph edges. the minimal df schema is: 4 id columns and at least one property to update

    +--------------------+--------------------+-------+--------------------+-------------------+
    |                 src|                 dst| ~label|                  id|               prop|
    +--------------------+--------------------+-------+--------------------+-------------------+
    |god:THxdAAAAAAAAAAAA|titan:J474AAAAAAA...| father|da0a9900-8fe1-11e...|              value|
    +--------------------+--------------------+-------+--------------------+-------------------+

    if ID column is not present it will be generated and edges will be saved as new.

    df

    data frame with edge ids and update columns

    cache

    cache df before processing, true by default for consistence updates. two C* entries need to be updated for one edge, so no reloads expected between this two calls.

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  58. def updateEdges(df: DataFrame): Unit

    Permalink

    shortcut for updateEdges(df: DataFrame, cache: Boolean = true) for Java

    shortcut for updateEdges(df: DataFrame, cache: Boolean = true) for Java

    Definition Classes
    DseGraphFrame
  59. def updateVertices(vertexLabel: String, df: DataFrame): Unit

    Permalink

    update this graph vertices with properties provided in the df.

    update this graph vertices with properties provided in the df. you should provide id in non encoded format

    +-----------------+---------+---------+
    |     community_id|member_id|      age|
    +-----------------+---------+---------+
    |       1182054400|        0|        0|
    +-----------------+---------+---------+

    the df is not cached by the function.

    vertexLabel

    to update

    df

    dataframe with vertex id and update columns

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  60. def updateVertices(df: DataFrame, labels: Seq[String] = Seq.empty, cache: Boolean = true): Unit

    Permalink

    update this graph vertices with properties provided in the df.

    update this graph vertices with properties provided in the df. the minimal df schema is just vertex "id" and one property to update:

    +-----------------+---------+
    |               id|      age|
    +-----------------+---------+
    |god:AAAAATMAAA...|        0|
    +-----------------+---------+

    label and vertices id will be extracted from the graph frame id. for better performance it is recommended to add/leave "~label" column

    +-----------------+---------+---------+
    |               id|   ~label|      age|
    +-----------------+---------+---------+
    |god:AAAAATMAAA...|      god|        0|
    +-----------------+---------+---------+

    you can also provide id in non encoded format

    +-----------------+---------+---------+---------+
    |     community_id|member_id|   ~label|      age|
    +-----------------+---------+---------+---------+
    |       1182054400|        0|      god|        0|
    +-----------------+---------+---------+---------+

    Note: passing both synthetic "id" and vertex Id columns is an error.

    df

    dataframe with vertex id and update columns

    labels

    empty (means all) by default, it is convenient to group vertexes with the same id format. That group could be passed here, to reduce number of verification steps

    cache

    cache df before processing, true by default for consistence update and performance

    Definition Classes
    ClassicDseGraphFrameDseGraphFrame
  61. def updateVertices(df: DataFrame): Unit

    Permalink

    shortcut for updateVertices(df: DataFrame, labels: Seq[String] = Seq.empty, cache: Boolean = true) for Java API

    shortcut for updateVertices(df: DataFrame, labels: Seq[String] = Seq.empty, cache: Boolean = true) for Java API

    df

    dataframe with vertex id and update columns

    Definition Classes
    DseGraphFrame
  62. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  63. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  64. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. def withCachedDf[T](df: DataFrame, cache: Boolean)(code: ⇒ T): T

    Permalink
    Attributes
    protected
    Definition Classes
    DseGraphFrame

Inherited from DseGraphFrame

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped