About DataStax Graph
DataStax Graph (DSG) is a real-time distributed graph database, tightly integrated in DataStax’s distribution of Apache Cassandra®. It is optimized for fast data storage and traversals, zero downtime, and analysis of complex, disparate, and related datasets. DataStax Graph is the graph model for Apache Cassandra that can scale to massive datasets and executing both transactional and analytical workloads. DataStax Graph incorporates all of the enterprise-class functionality found in DataStax Enterprise (DSE), including advanced security protection, built-in DSE Analytics and DSE Search functionality, DSE OpsCenter management UI and monitoring, and development tools including DataStax Studio.
DSE Graph is most beneficial when data is highly complex and connected, revealing depth and breadth of the relationships between entities. Use cases include:
-
Fraud detection
-
Recommendation engines
-
IT network and device management
-
Inventory management
-
Master data management
About graph databases
A graph database uses graph structures to store data along with the data’s relationships. Common use cases include fraud prevention, Customer 360, Internet of Things (IoT) predictive maintenance, and recommendation engines. Graph databases use a data model that is as simple as a whiteboard drawing with vertices, edges, and properties as described in Data modeling.
With DataStax Graph, graph data is stored in a DSE database, and the graph data model is designed to represent entities and their relationships efficiently and clearly. DSE Graph uses query optimization that automatically processes as much of the query as possible in parallel, leading to increased performance. Before starting a graph traversal, graph index structures provide optimal entry points for queries. During graph traversal, graph partitioning handles vertices with extreme connectedness to prevent hotspots.
Not all data is appropriate for storage in a graph database. Use DSE Graph to store data when the following characteristics are required:
-
Comprehensive data model
-
Data is database-centric within a single query
-
Entities and relationships are queried
-
Application is read heavy
Use the transactional (cassandra) DSE database to store data when the following characteristics are required:
-
Heavy denormalization
-
Data is application centric with multiple queries
-
Individual entities are queried
-
Application is write heavy
DataStax Graph features
Scalable for large graphs and high volumes of users, events, and operations |
DSG can contain billions (109) of vertices and edges. It takes advantage of the unique scalability of Apache Cassandra® to store graph data. The DSE database architecture can handle petabytes of information and thousands of concurrent users and operations per second. |
Support for high-volume, concurrent transactions and operational graph processing (OLTP) |
The transactional capacity of DSG scales with the size of the cluster and answers complex traversal queries on huge graphs in milliseconds. |
Support for global graph analytics and batch graph processing (OLAP) |
Available through the Apache Spark™ framework. |
Deep integration with Apache Cassandra |
DSG has been re-engineered to deeply integrate with Apache Cassandra, giving developers the flexibility to read and write graph data using CQL, Gremlin, or both, if desired. Graph data is written once and can be read with either API. |
Integration with DSE Search |
Integrates with DSE Search for efficient indexing that supports geographical and numeric range search, as well as full-text search for vertices and edges in large graphs. |
Native support for Apache TinkerPop™ and Gremlin query language |
Uses the popular property graph data model exposed by Apache TinkerPop and the graph traversal language Gremlin. Apache TinkerPop is an open source project that provides an abstraction framework to interact with DSG and other graph databases. Gremlin is the primary interface into DSG. Gremlin is a graph traversal language and virtual machine developed by Apache TinkerPop. Gremlin is a functional language that enables Gremlin to naturally support imperative and declarative querying. |
Automatic performance tuning |
More graph-level configuration is automatically tuned to decrease operational complexity. |
Vertex-centric indexes provide optimal querying |
Allows optimized deep traversal by quickly reducing search space. |
Optimized disk representation |
Allows for efficient use of storage and speed of access. DSG runs on commodity hardware with common specifications like other DSE offerings. See Capacity planning and hardware selection for DSE deployments. |
DataStax Graph versus other graph databases
DSG is distributed, highly available, and has a scale-out architecture. The data in DSG is automatically partitioned across all the nodes in a cluster like other Apache Cassandra data. Additionally, DSG has built-in support for OLAP tics and search of graph data. The advanced security options of DSE apply to all graph data.
With the redesign of DSG, three equally capable methods of data model schema creation now exist. Which method you use depends on your skill set and preferences. The methods are:
-
Use Gremlin exclusively to create a graph and schema and query the graph.
-
Use Cassandra Query Language (CQL) to create a graph and schema that can be queried with Gremlin.
-
Convert data stored in Apache Cassandra to a graph that can be queried with Gremlin.
DataStax Graph compared to relational databases
|
Best practices start with data modeling before inserting data. The paradigm shift between relational and graph databases requires careful sis of data and data modeling before importing and querying data in a graph database. For more information and examples, see DataStax Graph data modeling. |
Graph databases and relational databases can both store data, but the storage method, scalability, and indexing techniques that each uses are different.
- Storage
-
Both relational databases and DSE Graph store data in tables. In a relational database, a key is defined to retrieve data, and a foreign key is required to link tables that share a relationship. Data is normalized, which means that the data is stored so that no duplication occurs. To write complex queries, several tables must be accessed to join and retrieve data.
DSE Graph stores data in the DSE database tables, and the relationship between data is embedded in the model. Data storage in a graph database can be compared to a pre-joined relational database, with built-in data relationships, so foreign keys are unnecessary. Data is retrieved by traversing the graph, so time-consuming and error-prone
JOINoperations are not required. - Scaling
-
An added bonus for graph databases is the ability to scale database distribution. A graph database query starts with a particular vertex and filters the graph based on the query requirements. Because the retrieval generally involves filtering subset(s) of the entire graph, the graph can be partitioned. The query can run in parallel for some steps of the traversal, operating on either a single or small number of nodes in the DSE cluster. Because of interconnected tables, successful queries require accessing many tables in relational databases. Joining data is costly, which makes relational databases hard to scale out and distribute.
- Indexing
-
Indexing techniques between graph databases and relational databases are distinct and can affect performance markedly. Consider the query for looking up all buyers that purchased an Xbox One.
In a relational database, three tables would be accessed to answer the query:
-
Each buyer in the
buyerstable -
The item being queried in the
itemstable -
The buyer and item in the
buyers-itemstable, created through a JOIN operation
In contrast, a graph database requires only one index lookup for the item, and then traverses the graph to each buyer vertex to complete the query.
The graph database query will run in near-constant time. The relational database query, if a balanced tree (B-tree) index is assumed, runs in O(log N) time, where N is the number of records in the table for each foreign key that must be referenced. Even a one-deep query will greatly impact performance; if a query goes to a more deeply nested depth, the performance gap will further increase.
-
- Query language
-
Queries in DSE Graph are expressed using the Gremlin traversal language. Writing complex queries in Gremlin is simpler than in relational databases using SQL. The following example compares SQL and Gremlin queries for a product recommendation engine using collaborative filtering ("customers who bought this also bought…").
In SQL, collaborative filtering requires joining a transactional table to itself:
WITH target_user_purchases AS ( -- Step 1: Get all products already purchased by the target user SELECT product_id FROM orders WHERE user_id = 'user_123' ), similar_users AS ( -- Step 2: Find peers who bought at least one of the same products SELECT o.user_id, COUNT(*) as overlap_count FROM orders o JOIN target_user_purchases t ON o.product_id = t.product_id WHERE o.user_id <> 'user_123' GROUP BY o.user_id ), recommended_products AS ( -- Step 3: Find what else those similar users bought SELECT o.product_id, SUM(s.overlap_count) as recommendation_score FROM orders o JOIN similar_users s ON o.user_id = s.user_id -- Step 4: Exclude products the target user already owns WHERE o.product_id NOT IN (SELECT product_id FROM target_user_purchases) GROUP BY o.product_id ) -- Step 5: Return top 5 recommendations with product details SELECT p.product_id, p.product_name, r.recommendation_score FROM recommended_products r JOIN products p ON r.product_id = p.product_id ORDER BY r.recommendation_score DESC LIMIT 5;With Gremlin, collaborative filtering is expressed as an intuitive traversal where each step progressively follows relationships across the graph:
g.V('customerId','ALFKI').as('customer') \ .out('ordered').out('contains').out('is').as('products') \ .in('is').in('contains').in('ordered').except('customer') \ .out('ordered').out('contains').out('is').except('products') \ .groupCount().cap().orderMap(T.decr)[0..<5].productName
Connect to DataStax Graph
-
DataStax Bulk Loader (DSBulk): A command line utility that supports loading CSV and JSON data files.
-
DataStax Studio: A web-based interactive developer tool with notebooks for running Gremlin commands and visualizing graphs, including creating graph schemas, loading data, and traversing the graph. DataStax Studio provides both tabular and visual information for DSG schema and queries, enhancing the exploration of graph relationships.
-
Gremlin console: A shell utility for loading, querying, and exploring graph data using graph traversals.
-
DseGraphFrame: A framework for the Apache Spark™ API that can load data to DSG directly or with transformations.
-
Apache Cassandra drivers: For application development, pass Gremlin statements to DSE Graph.