Configuring Apache Spark™
Configuring Spark for DataStax Enterprise includes:
- Configuring Spark nodes
Modify the settings for Spark nodes security, performance, and logging.
- Automatic Spark Master election
Spark Master elections are automatically managed.
- Configuring Spark logging options
Configure Spark logging options.
- Running Spark processes as separate users
Spark processes can be configured to run as separate operating system users.
- Configuring the Spark history server
Load the event logs from Spark jobs that were run with event logging enabled.
- Enabling Spark apps in cluster mode when authentication is enabled
Configuration steps to enable Spark applications in cluster mode when JAR files are on the Cassandra file system (CFS) and authentication is enabled.
- Setting Spark Cassandra Connector-specific properties
Use the Spark Cassandra Connector options to configure DataStax Enterprise Spark.
- Creating a DSE Analytics Solo datacenter
DSE Analytics Solo datacenters do not store any database or search data, but are strictly used for analytics processing. They are used in conjunction with one or more datacenters that contain database data.
- Spark JVMs and memory management
Spark jobs running on DataStax Enterprise are divided among several different JVM processes.