Mapping rules for `DseGraphFrame`

DseGraphFrame uses mapping rules for column names and types.

Column mapping rules

DataFrame column names are the same as graph property names except in the following cases.

Conflict with column names reserved by GraphFrame will result in an underscore (_) added to the property name. For example, the id column will result in a property named _id.

Reserved column name
Reserved column name
`id`
`src`
`dst`
`new_id`
`new_src`
`new_dst`
`graphx_attr`

DseGraphFrame and Spark SQL are case insensitive by default. Column names that differ only in case will result in conflicts. Set the Spark property spark.sql.caseSensitive=true to avoid case conflicts.
```
$ dse spark --conf spark.sql.caseSensitive=true
```

Type mapping rules

DseGraphFrame and Spark SQL have a limited set of supported types. A vertex is represented by a Row instance.

If the vertex has multiple properties, each property will be represented as a Spark SQL array with property values. If a property has meta-properties it will be represented as StructType. The value field of the struct contains the property value. All other fields will represent the meta-properties.

DSE Graph to Spark SQL and DseGraphFrame type mapping
DSE Graph type	Spark SQL type	Conversion rules
`boolean`	`BooleanType`
`smallint`	`ShortType`
`int`	`IntegerType`
`bigint`	`LongType`
`float`	`FloatType`
`double`	`DoubleType`
`decimal`	`DecimalType(38, 18)`
`varint`	`DecimalType(38, 0)`
`timestamp`	`TimestampType`
`date`	`DateType`
`time`	`LongType`	The number of nanoseconds from the beginning of the day.
`text`	`StringType`
`uuid`	`StringType`	The `UUID.toString()` and `UUID.fromString()` methods are used to convert the value.
`inet`	`StringType`	The `toString` and `InetAddress.getByName()` methods are used to convert the value.
`blob`	`BinaryType`
`'PointType'`	`StringType`	The `toWellKnownText()` and `fromWellKnownText()` methods are used to convert the value.
`'LineStringType'`	`StringType`	The `toWellKnownText()` and `fromWellKnownText()` methods are used to convert the value.
`'PolygonType'`	`StringType`	The `toWellKnownText()` and `fromWellKnownText()` methods are used to convert the value.

Mapping rules for `DseGraphFrame`

Column mapping rules

Type mapping rules

Was this helpful?

Give Feedback

Mapping rules for DseGraphFrame

Column mapping rules

Type mapping rules

Was this helpful?

Mapping rules for `DseGraphFrame`