Accessing DataStax Enterprise data from external Spark clusters

DataStax Enterprise works with external Spark clusters in a bring-your-own-Spark (BYOS) model.

Overview of BYOS support in DataStax Enterprise

DataStax Enterprise provides a JAR and configuration files for connected to DataStax Enterprise clusters from external Spark clusters.

Generating the BYOS configuration file

The byos.properties file contains configuration settings to connect to a particular DataStax Enterprise cluster.

Connecting to DataStax Enterprise using the Spark shell on an external Spark cluster

Use the Spark shell on an external Spark cluster to connect to DataStax Enterprise

Generating Spark SQL schema files

Generate Spark SQL schema files for use with Spark SQL on external Spark clusters.

Starting Spark SQL Thrift Server with Kerberos

Starting Spark SQL Thrift Server with Kerberos and BYOS.

Accessing HDFS or CFS resources using Kerberos authentication

HDFS or CFS resources can be accessed from BYOS nodes using Kerberos authentication.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com