graphframe

The root package of DSE GraphFrame support. Offers handy implicit conversions that add DSE-specific methods to SparkSession and GraphFrame.

Call dseGraph method on the SparkSession object to create a GraphFrame exposing DSE Graph as Spark GraphFrame.

Call GraphFrameFunctions saveToDse function on any GraphFrame to save updated graph back to DSE.

Example:

 import com.datastax.bdp.graph.spark.graphframe._

val graphName = "test"
// Read graph and print vertices labels:
 val graph = spark.dseGraph(graphName)
 graph.V().select("label").distinct.show

Linear Supertypes

AnyRef, Any

Type Members

class DGFCleaner extends AnyRef
class DGFCleanerInterceptor extends SparkVertexProgramInterceptor[CleanGraphVertexProgram]
class DGFCleanupStats extends AnyRef
class DseGraphFrame extends Serializable

Provides DSEGraph-specific methods on GraphFrame It support graphName is needed for some traversal steps and to write data back It can be lost during DseGraphFrame->GraphFrame->DseGraphFrame implicit conversions set graphName tot he target graph if needed.
class DseGraphFrameBuilder extends AnyRef

Helper class to create GraphFrame from C* backend.
Helper class to create GraphFrame from C* backend. The GraphFrame caches dataframes on creation. It is recommended to call withVertex method to create subgraph prior graph frame creation to reduce memory footprint
Usage: val dataFrame = DseGraphFrameBuilder("graph", spark).dseGraph()
class DseGraphTraversal[E] extends DefaultGraphTraversal[Row, E]

wrap vertex data frame to implement TinkerPop functions
type DseGraphTraversalInterceptor = PartialFunction[DseGraphTraversal[_], DataFrame]

should be used to implement optimization for DseGraphTraversal
should be used to implement optimization for DseGraphTraversal
if isDefinedAt(traversal: DseGraphTraversal[_]) methods returns true apply() methods return result DataFrame. It will be used as result instead of standard DseGraphTraversal enumerates internal list of interceptors. TODO: provide user faced extension mechanism
class EdgeBuilder extends GraphComponentBuilder with Serializable
abstract class GraphComponentBuilder extends Serializable
class SparkSessionFunctions extends Serializable

Provides DSE-specific methods on SparkSession
class VertexBuilder extends GraphComponentBuilder with Serializable

Internal class to build table for one class of vertex

Value Members

object DGFCleaner
object DseGraphFrame extends Serializable
object DseGraphFrameBuilder

Usage: val dataFrame = DseGraphFrameBuilder("graph", spark).dataFrame()
object Order

spark-shell show error: "ambiguous reference to overloaded definition" for two functions: GraphTraversal.by(Order) GraphTraversal.by(Comparable) * scala> g.V().values("name").order().by(Order.decr) console>:43: error: ambiguous reference to overloaded definition, both method by in trait GraphTraversal of type (x$1: org.apache.tinkerpop.gremlin.process.traversal.Order)org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal[org.apache.spark.sql.Row,E2] and method by in trait GraphTraversal of type (x$1: java.util.Comparator[E2])org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal[org.apache.spark.sql.Row,E2] match argument types (org.apache.tinkerpop.gremlin.process.traversal.Order)
spark-shell show error: "ambiguous reference to overloaded definition" for two functions: GraphTraversal.by(Order) GraphTraversal.by(Comparable) * scala> g.V().values("name").order().by(Order.decr) console>:43: error: ambiguous reference to overloaded definition, both method by in trait GraphTraversal of type (x$1: org.apache.tinkerpop.gremlin.process.traversal.Order)org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal[org.apache.spark.sql.Row,E2] and method by in trait GraphTraversal of type (x$1: java.util.Comparator[E2])org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal[org.apache.spark.sql.Row,E2] match argument types (org.apache.tinkerpop.gremlin.process.traversal.Order)
To workaround this problem you should directly cast Order to super class: Comparable scala> g.V().values("name").order().by(Order.decr.asInstanceOf[Order])
that is inconvenient for user. This class is replicate the Order interface but returns Comparable instread of Order. It is used in spark-shell imports
object VertexLabelDropInterceptor extends DseGraphTraversalInterceptor
implicit def backToDseVertexFrame[E](gt: GraphTraversal[Row, E]): DseGraphTraversal[E]
implicit def fromDseGraphFrame(gf: DseGraphFrame): GraphFrame
implicit def fromVertexFrame[E](gt: GraphTraversal[Row, E]): DataFrame

add all DataFrame functions to DseGraphTraversal
package optimizer
implicit def toDseGraphFrame(gf: GraphFrame): DseGraphFrame
implicit def toSparkSessionFunctions(spark: SparkSession): SparkSessionFunctions

package graphframe

Type Members

class DGFCleaner extends AnyRef

class DGFCleanerInterceptor extends SparkVertexProgramInterceptor[CleanGraphVertexProgram]

class DGFCleanupStats extends AnyRef

class DseGraphFrame extends Serializable

class DseGraphFrameBuilder extends AnyRef

class DseGraphTraversal[E] extends DefaultGraphTraversal[Row, E]

type DseGraphTraversalInterceptor = PartialFunction[DseGraphTraversal[_], DataFrame]

class EdgeBuilder extends GraphComponentBuilder with Serializable

abstract class GraphComponentBuilder extends Serializable

class SparkSessionFunctions extends Serializable

class VertexBuilder extends GraphComponentBuilder with Serializable

Value Members

object DGFCleaner

object DseGraphFrame extends Serializable

object DseGraphFrameBuilder

object Order

object VertexLabelDropInterceptor extends DseGraphTraversalInterceptor

implicit def backToDseVertexFrame[E](gt: GraphTraversal[Row, E]): DseGraphTraversal[E]

implicit def fromDseGraphFrame(gf: DseGraphFrame): GraphFrame

implicit def fromVertexFrame[E](gt: GraphTraversal[Row, E]): DataFrame

package optimizer

implicit def toDseGraphFrame(gf: GraphFrame): DseGraphFrame

implicit def toSparkSessionFunctions(spark: SparkSession): SparkSessionFunctions

Inherited from AnyRef

Inherited from Any

Ungrouped