Running spark-submit job with internal authentication

Example of running a spark-submit job with internal authentication.

This example shows how to run a spark-submit job with internal authentication.

When you use dse spark-submit to submit a Spark job, the Spark Master URL and the Spark database connection URL are set automatically. Use the Spark session builder API to set the application name. For example:

SparkSession spark = SparkSession
      .builder()
      .appName("Datastax Java example")
      .getOrCreate();

Clone the example source files from github.

git clone https://github.com/datastax/SparkBuildExamples.git

Select you preferred language and build system. For example for Java and Maven:
```
cd SparkBuildExamples/java/maven/dse
```
Build the package with Maven:
```
mvn package
```
Create your authentication credentials.

Authentication credentials can be provided in several ways, see Providing credentials from DSE tools.
Use spark-submit to run the application. The following example assumes you’ve set your authentication credentials in an environment variable or config file.
```
dse spark-submit --class com.datastax.spark.example.WriteRead ./target/writeRead-0.1.jar
```

Running spark-submit job with internal authentication

Was this helpful?

Give Feedback