Spark SSL encryption 

Communication between Spark clients and clusters as well as communication between Spark nodes can be encrypted using SSL.

Communication between Spark clients and clusters as well as communication between Spark nodes can be encrypted using SSL. You must configure encryption on each node in your cluster.

Configure Spark SSL encryption on the server-side by editing dse.yaml, and for Spark clients by editing spark-defaults.conf in the Spark configuration directory.

The default location of the Spark configuration files depends on the type of installation:
Installer-Services and Package installations /etc/dse/spark/
Installer-No Services and Tarball installations install_location/resources/spark/conf/

Spark SSL encryption is limited to Akka control messages and file sharing. It does not encrypt RDD data exchanges or the web user interface.

Note: Using Spark with SSL encryption might result in a slight drop in Spark job performance because of the extra time to encrypt and decrypt the data and perform the extra garbage collection (GC).
The location of the dse.yaml file depends on the type of installation:
Installer-Services /etc/dse/dse.yaml
Package installations /etc/dse/dse.yaml
Installer-No Services install_location/resources/dse/conf/dse.yaml
Tarball installations install_location/resources/dse/conf/dse.yaml

Procedure

  1. Open dse.yaml in a text editor.
  2. In the spark_encryption_options section set the options as described below.
    Option Description
    enabled Enables or disables server-side encryption. The default is false.
    keystore The path to the keystore file, relative to the Spark configuration directory. The default keystore is a file named .keystore located in the Spark configuration directory.
    keystore_password The password used to access the keystore. The default password is cassandra.
    truststore The path to the truststore file, relative to the Spark configuration directory. The default truststore is a file named .truststore located in the Spark configuration directory.
    truststore_password The password used to access the truststore. The default password is cassandra.
    protocol The SSL protocol used when encrypting communications. The default is TLS.
    cipher_suites The cipher suites used with the protocol, enclosed in square brackets ([]) and separated by commas. The default suites are [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA].
  3. In each client, set the client encryption options in the spark-default.conf file in the Spark configuration directory.
    Option Description
    spark.ssl.enabled Enables or disables client-side encryption. The default is false.
    spark.ssl.keyStore The path to the keystore file, relative to the Spark configuration directory. The default keystore is a file named .keystore located in the Spark configuration directory.
    spark.ssl.keyStorePassword The password used to access the keystore. The default password is cassandra.
    spark.ssl.keyPassword The password for the private key. The default password is cassandra.
    spark.ssl.trustStore The path to the truststore file, relative to the Spark configuration directory. The default truststore is a file named .truststore located in the Spark configuration directory.
    spark.ssl.trustStorePassword The password used to access the truststore. The default password is cassandra.
    spark.ssl.protocol The SSL protocol used when encrypting communications. The default is TLS.
    spark.ssl.enabledAlgorithms The cipher suites used with the protocol, separated by commas. The default suites are TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA.
    spark.ssl.useNodeLocalConf Sets whether the Spark executors inherit the SSL configuration from the Spark Workers. The default is true.