Loading Gryo data

One file format for importing and exporting data to and from DSE Graph is Gryo, a binary file format. Gryo is a Gremlin variant of Kryo, a fast and efficient object graph serialization framework for Java.

The data mapping script for Gryo data is shown with explanation. The full script is found at the bottom of the page.

DSE Graph Loader can load Gryo files generated with DSE Graph or with TinkerGraph, the in-memory graph database included with Apache TinkerPop. The Gryo files generated with DSE Graph have a different format from TinkerGraph Gryo files, and the mapping script is differentfor loading data from each source.

If desired, add configuration to the mapping script.

Specify the data input file. The variable inputfiledir specifies the directory for the input file. The identified file will be used for loading.

// DATA INPUT
// Define the data input source
// inputfiledir is the directory for the input files

inputfiledir = '/tmp/Gryo/'
recipeInput = Graph.file(inputfiledir + 'recipe.gryo').gryo()

If the Gryo input file is generated from DSE Graph, an additional step dse() will allow the input data to be streamed, facilitating large file transfers.

// DATA INPUT
// Define the data input source
// inputfiledir is the directory for the input files

inputfiledir = '/tmp/Gryo/'
recipeInput = Graph.file(inputfiledir + 'recipe.gryo').gryo().dse()

The file is specified as a gryo file and an additional step gryo() identifies that the file should be processed as a Gryo file. A map, recipeInput, is created that will be used to process the data.

recipeInput = Graph.file(inputfiledir + 'recipe.gryo')

Note that Graph.file is used, in contrast to File.csv or File.json.

If you wish to access a java.io.File object, fully namespace the first call; otherwise, DSE Graph Loader overrides the File object:

currentDir = new java.io.File('.').getCanonicalPath() + '/'
source = Graph.file(currentDir + 'myfile.kryo').gryo()

Create the main body of the mapping script. This part of the mapping script is the same regardless of the file format, although Gryo files use a slightly modified version.
To run DSE Graph Loader for Gryo loading as a dry run, use the following command:
```
$ graphloader recipeMappingGRYO.groovy -graph testGRYO -address localhost -dryrun true
```
For testing purposes, the graph specified does not have to exist prior to running graphloader. However, for production applications, the graph and schema should be created prior to using graphloader.

The full loading script is shown:

/* SAMPLE INPUT
Gryo file is a binary file
 */

// CONFIGURATION
// Configures the data loader to create the schema
config create_schema: true, load_new: true

// DATA INPUT
// Define the data input source
// inputfiledir is the directory for the input files

inputfiledir = '/tmp/GRYO/'
recipeInput = Graph.file(inputfiledir + 'recipe.gryo').gryo()

load(recipeInput.vertices()).asVertices {
    labelField "~label"
    key "~id", "id"
}

load(recipeInput.edges()).asEdges {
    labelField "~label"
    outV "outV", {
        labelField "~label"
        key "~id", "id"
    }
    inV "inV", {
        labelField "~label"
        key "~id", "id"
    }
}

Loading Gryo data

Was this helpful?

Give Feedback