• Glossary
  • Support
  • Downloads
  • DataStax Home
Get Live Help
Expand All
Collapse All

DataStax Enterprise 5.1 Documentation

    • Overview
      • Release notes
      • New features
    • Architecture
      • Architecture FAQ
      • Database architecture
        • Architecture in brief
        • Internode communications (gossip)
        • Data distribution and replication
          • Data distribution overview
          • Consistent hashing
          • Virtual nodes
            • Data replication
          • Partitioners
          • Snitches
            • Dynamic snitching
            • Types of snitches
        • Node repair
          • Hinted handoff: repair during write path
          • Read Repair: repair during read path
          • Anti-entropy repair
      • Component architecture
        • DSE Analytics
        • DSE Search
        • DSE Graph
          • When to use DSE Graph
          • DSE Graph, OLTP, and OLAP
          • Comparing DSE Graph and relational databases
          • Migrating to DSE Graph from a relational database
          • Migrating to DSE Graph from Apache Cassandra
      • Database internals
        • Storage engine
        • About reads and writes
          • How is data written?
          • How is data maintained?
          • How is data updated?
          • How is data deleted?
          • What are tombstones?
          • How are indexes stored and updated?
          • How is data read?
          • How do write patterns affect reads?
        • Data consistency
          • Read and write consistency
          • Differences between DSE and RDMBS transactions
          • Using lightweight transactions
          • Consistency level performance
          • Consistency level configuration
          • Configuring serial consistency
          • Read requests
            • Examples of read consistency levels
          • Write requests
            • Multiple datacenter write requests
    • Planning
      • Initializing a cluster
        • Initializing datacenters
          • Initializing a single datacenter per workload type
          • Initializing multiple datacenters per workload type
        • Initializing single-token architecture datacenters
          • Calculating tokens for single-token architecture nodes
    • Getting started
    • Installing DSE
      • Which install method should I use?
      • DataStax Installers
        • DataStax Installer (root permissions)
        • DataStax Installer (no root permissions)
        • DataStax Installer (unattended)
      • Lifecycle Manager
      • Package installer using Yum
      • RedHat systemd configuration
      • Package installer using APT
      • Binary tarball installer
      • Installing DSE patch releases
      • Installing on cloud providers
      • Installing on Docker
      • Installing supporting software
      • Uninstalling DSE
      • Installing CQLSH
      • Default file locations
        • Package and Installer-Services installations
        • Tarball and Installer-No Services installations
    • Managing
      • Configuration
        • Recommended production settings
        • YAML and configuration properties
          • cassandra.yaml
          • dse.yaml
          • remote.yaml
          • cassandra-rackdc.properties
          • cassandra-topology.properties
        • Configuring snitches for cloud providers
          • Ec2Snitch
          • Ec2MultiRegionSnitch
          • GoogleCloudSnitch
          • CloudstackSnitch
        • Start-up parameters
        • Choosing a compaction strategy
        • Using multiple network interfaces
        • Configuring gossip directory
        • Configuring heap dump directory
        • Configuring Virtual Nodes
          • Virtual node (vnode) configuration
          • Enabling virtual nodes on an existing production cluster
        • Logging configuration
          • Changing logging locations
          • Configuring logging
          • Commit log archive configuration
          • Change Data Capture (CDC) logging
      • Tools
        • DSE Metrics Collector
        • nodetool
          • About the nodetool utility
          • abortrebuild
          • assassinate
          • bootstrap
          • cfhistograms
          • cfstats
          • cleanup
          • clearsnapshot
          • compact
          • compactionhistory
          • compactionstats
          • decommission
          • describecluster
          • describering
          • disableautocompaction
          • disablebackup
          • disablebinary
          • disablegossip
          • disablehandoff
          • disablehintsfordc
          • disablethrift
          • drain
          • enableautocompaction
          • enablebackup
          • enablebinary
          • enablegossip
          • enablehandoff
          • enablehintsfordc
          • enablethrift
          • failuredetector
          • flush
          • garbagecollect
          • getcachecapacity
          • getcachekeystosave
          • gcstats
          • getcompactionthreshold
          • getcompactionthroughput
          • getconcurrentcompactors
          • getendpoints
          • getinterdcstreamthroughput
          • getlogginglevels
          • getseeds
          • getsstables
          • getstreamthroughput
          • gettimeout
          • gettraceprobability
          • gossipinfo
          • help
          • gethintedhandoffthrottlekb
          • info
          • invalidatecountercache
          • invalidatekeycache
          • invalidaterowcache
          • join
          • listendpointspendinghints
          • listsnapshots
          • mark_unrepaired
          • move
          • netstats
          • pausehandoff
          • proxyhistograms
          • rangekeysample
          • rebuild
          • rebuild_index
          • rebuild_view
          • refresh
          • refreshsizeestimates
          • reloadlocalschema
          • reloadtriggers
          • reloadseeds
          • relocatesstables
          • removenode
          • repair
          • replaybatchlog
          • resetlocalschema
          • resume
          • resumehandoff
          • ring
          • scrub
          • sequence
          • setcachecapacity
          • setcachekeystosave
          • setcompactionthreshold
          • setcompactionthroughput
          • setconcurrentcompactors
          • sethintedhandoffthrottlekb
          • setinterdcstreamthroughput
          • setlogginglevel
          • setstreamthroughput
          • settimeout
          • settraceprobability
          • sjk
          • snapshot
          • status
          • statusbackup
          • statusbinary
          • statusgossip
          • statushandoff
          • statusthrift
          • stop
          • stopdaemon
          • tablehistograms
          • tablestats
          • toppartitions
          • tpstats
          • truncatehints
          • upgradesstables
          • verify
          • version
          • viewbuildstatus
        • dse commands
          • About dse commands
          • dse connection options
          • add-node
          • beeline
          • cassandra
          • cassandra-stop
          • exec
          • fs
          • gremlin-console
          • hadoop fs
          • list-nodes
          • pyspark
          • remove-node
          • spark
          • spark-class
          • spark-jobserver
          • spark-history-server
          • spark-sql
          • spark-sql-thriftserver
          • spark-submit
          • SparkR
          • -v
        • dse client-tool
          • About dse client-tool
          • client-tool connection options
          • cassandra
          • configuration export
          • configuration byos-export
          • configuration import
          • spark
        • dsetool
          • About dsetool
          • dsetool connection options
          • autojt
          • checkcfs
          • core_indexing_status
          • create_core
          • createsystemkey
          • encryptconfigvalue
          • get_core_config
          • get_core_schema
          • help
          • index_checks
          • infer_solr_schema
          • inmemorystatus
          • insights_config
          • insights_filters
          • list_core_properties
          • list_index_files
          • list_subranges
          • managekmip list
          • managekmip expirekey
          • managekmip revoke
          • managekmip destroy
          • node_health
          • partitioner
          • perf
          • read_resource
          • rebuild_indexes
          • repaircfs
          • reload_core
          • ring
          • set_core_property
          • sparkmaster cleanup
          • sparkworker restart
          • status
          • stop_core_reindex
          • tieredtablestats
          • tsreload
          • unload_core
          • upgrade_index_files
          • write_resource
        • SSTable utilities
          • sstabledump
          • sstableexpiredblockers
          • sstablelevelreset
          • sstableloader
          • sstablemetadata
          • sstableofflinerelevel
          • sstablepartitions
          • sstablerepairedset
          • sstablescrub
          • sstablesplit
          • sstableupgrade
          • sstableutil
          • sstableverify
        • Preflight check tool
        • cluster_check and yaml_diff tools
      • Operations
        • Starting and stopping DSE
          • Starting as a service
          • Starting as a stand-alone process
          • Stopping a node
        • Adding or removing nodes, datacenters, or clusters
          • Adding vnodes to an existing cluster
          • Adding a datacenter to a cluster
          • Adding a datacenter to a cluster using a designated datacenter as a data source
          • Replacing a dead node or dead seed node
          • Replacing a running node
            • Adding a node and then decommissioning the old node
            • Replacing a running node
          • Moving a node from one rack to another
          • Decommissioning a datacenter
          • Removing a node
          • Changing the IP address of a node
          • Switching snitches
          • Changing keyspace replication strategy
          • Migrating or renaming a cluster
          • Adding single-token nodes to a cluster
          • Adding a datacenter to a single-token architecture cluster
          • Replacing a dead node in a single-token architecture cluster
        • Backing up and restoring data
          • About snapshots
          • Taking a snapshot
          • Deleting snapshot files
          • Enabling incremental backups
          • Restoring from a snapshot
          • Restoring a snapshot into a new cluster
          • Recovering from a single disk failure using JBOD
        • Repairing nodes
          • Manual repair: Anti-entropy repair
          • When to run anti-entropy repair
          • Changing repair strategies
            • Migrating to full repairs
            • Migrating to incremental repairs
        • Monitoring a DSE cluster
        • Tuning the database
          • Tuning Java resources
            • Changing heap size parameters
            • Configuring the garbage collector
              • G1 MaxGCPauseMillis
              • CMS parameters
          • Tuning Bloom filters
          • Configuring memtable thresholds
        • Data caching
          • Configuring data caches
            • Enabling and configuring caching
            • Tips for efficient cache use
          • Monitoring and adjusting caching
        • Compacting and compressing
          • Configuring compaction
          • Compression
            • When to compress data
            • Configuring compression
          • Testing compaction and compression
        • Migrating data to DSE
        • Collecting node health and indexing scores
        • Clearing data from DataStax Enterprise
      • DSE Management Services
        • DSE Performance Service
          • Performance Service
          • Configuring Performance Service replication strategy
          • Collecting data
            • Collecting slow queries
            • Collecting system level diagnostics
            • Collecting object I/O level diagnostics
            • Statistics gathered for objects
            • Collecting database summary diagnostics
            • Collecting cluster summary diagnostics
            • Collecting histogram diagnostics
            • Collecting user activity diagnostics
            • Statistics gathered for user activity
          • Collecting search data
            • Collecting slow search queries
            • Collecting indexing errors
            • Collecting Apache Solr performance statistics
            • Collecting cache statistics
            • Collecting index statistics
            • Collecting handler statistics
            • Collecting request handler metrics
          • Monitoring Spark with Spark Performance Objects
          • Diagnostic table reference
          • Solr diagnostic table reference
            • Frequently asked questions
            • Slow sub-query log for search
            • Indexing error log
            • Query latency snapshot
            • Update latency snapshot
            • Commit latency snapshot
            • Merge latency snapshot
            • Filter cache statistics
            • Query result cache statistics
            • Index statistics
            • Update handler statistics
            • Update request handler statistics
            • Search request handler statistics
      • DSE In-Memory
        • Creating or altering tables to use DSE In-Memory
        • Verifying table properties
        • Managing memory
        • Backing up and restoring data
      • DSE Tiered Storage
        • About DSE Tiered Storage
        • Configuring DSE Tiered Storage
        • Testing configurations
      • DSE Multi-Instance
        • About DSE Multi-Instance
        • DSE Multi-Instance architecture
        • Adding nodes to DSE Multi-Instance
        • DSE Multi-Instance commands
    • Securing
      • Security FAQs
      • Security checklists
      • Securing the environment
        • Securing ports
        • Securing the TMP directory
      • Configuring DSE Unified Authentication
        • About DSE Unified Authentication
          • Implementing DSE Unified Authentication
          • Implementing without downtime in production
        • Setting security keyspaces replication factors
        • Setting up Kerberos
          • Kerberos guidelines
          • Enabling JCE Unlimited
            • Removing AES-256
          • Preparing DSE nodes for Kerberos
        • Enabling DSE Unified Authentication
          • Defining a Kerberos scheme
          • Defining an LDAP scheme
        • Configuring JMX authentication
        • Managing credentials, role, and permissions cache settings
      • Connecting to authentication enabled clusters
        • About client connections
        • Providing credentials with DSE tools
        • Providing credentials with nodetool
        • Providing credentials with JConsole
        • Providing credentials with cqlsh
        • Using dsetool with Kerberos enabled cluster
        • Using cqlsh with Kerberos or user authentication
        • Loading data into a remote Kerberos enabled cluster
        • Graph and gremlin-console
        • Running Spark jobs with Kerberos
      • Managing roles
        • About roles
        • Creating superuser accounts
        • Creating roles for internal mode
        • Creating roles for LDAP mode
        • Creating roles for Kerberos principals
        • Binding a role to an authentication scheme
        • Configuring proxy roles for applications
      • Authorizing access to database resources
        • About permissions
        • Managing keyspace and table permissions
        • Setting row-level permissions
        • Managing access to DSE Graph keyspaces
        • Authorizing remote procedure calls for CQL execution
        • JMX MBean permissions
        • Search index permissions
        • Managing Spark application permissions
      • Auditing activity
        • Enabling data auditing
        • Configuring audit logging
          • Log formats
        • Configuring audit logging to a database table
          • CassandraAuditWriter table columns
        • Configuring auditing for DSE Search
      • Transparent data encryption
        • About Transparent Data Encryption
        • Configuring local encryption
          • Setting up local encryption keys
          • Encrypting configuration file properties
          • Encrypting system resources
          • Encrypting tables
          • Rekeying existing data
          • Troubleshooting encryption key errors
        • Configuring KMIP encryption
        • Encrypting Search indexes
          • Encrypting new Search indexes
          • Encrypting existing Search indexes
          • Tuning encrypted Search indexes
        • Migrating encrypted tables from earlier versions
        • Bulk loading data between TDE-enabled clusters
      • Configuring SSL
        • About SSL
        • Setting up SSL certificates
        • Securing internal transactional node connections
        • Securing client to cluster connections
        • Securing Spark connections
        • Using CQL shell (cqlsh) with SSL
        • Setting up SSL for nodetool, dsetool, and dse advrep
        • Setting up SSL for jconsole (JMX)
        • Connecting sstableloader to a secured cluster
        • Enabling SSL encryption for DSEFS
      • Dynamically set LDAP Authenticator Connection Search Password
    • Tooling Resources
      • Stress tools
        • The cassandra-stress tool
        • Interpreting the output of cassandra-stress
        • cfs-stress tool
      • OpsCenter services
        • Best Practice Service
        • Capacity Service
        • Repair Service
    • DSE Advanced Replication
      • About DSE Advanced Replication
      • Architecture
      • Traffic between the clusters
      • Terminology
      • Getting started
      • Keyspaces
      • Data Types
      • Operations
      • CQL queries
      • Metrics
      • Managing invalid messages
      • Managing audit logs
      • Command line tool
        • dse advrep commands
          • About the dse advrep command
          • channel create
          • channel update
          • channel delete
          • channel pause
          • channel resume
          • channel status
          • channel truncate
          • conf list
          • conf remove
          • conf update
          • destination create
          • destination update
          • destination delete
          • destination list
          • destination list-conf
          • destination remove-conf
          • metrics list
          • replog count
          • replog analyze-audit-log
    • DSE Analytics
      • Setting the replication factor for analytics keyspaces
      • DSE Analytics and Search integration
        • Using predicate push down in Spark SQL
      • About DSE Analytics Solo
      • DSEFS (DataStax Enterprise file system)
        • About DSEFS
        • Enabling DSEFS
        • Disabling DSEFS
        • Configuring DSEFS
        • Commands DSEFS
        • DSEFS compression
        • DSEFS authentication
        • DSEFS authorization
        • Using the DSEFS REST interface
        • Copying data from CFS to DSEFS
        • Programmatic access to DSEFS
        • Hadoop FileSystem interface implemented by DseFileSystem
        • Using JMX to read DSEFS metrics
      • Cassandra File System (deprecated)
    • DSE Graph
      • DSE Graph Terminology
      • Using JMX to read and execute operation with DSE Graph metrics
      • DSE Graph Configuration
        • Configuring DSE Graph options in the dse.yaml file
        • Configuring the Gremlin console in the remote.yaml
        • Configuring the Gremlin Server in the dse.yaml file
        • Configuring the Graph sandbox
        • Specifying the schema mode
        • Specifying DSE database and graph settings
        • Configuring DSE Graph Security
      • DSE Graph Tools
      • DSE Graph Reference
        • The schema API
          • clear
          • connection
          • config
          • describe
          • edgeLabel
          • exists
          • index - edge index
            • index - property index
            • index - vertex index
            • partitionKey - clusteringKey
            • properties
            • propertyKey
            • vertexLabel
          • The system API
          • create
          • drop
          • exists
          • graphs
          • option
          • replication
          • systemReplication
          • truncate
    • DSE Search
      • About DSE Search
        • DSE Search vs. OSS
        • Unsupported features for DSE Search
        • Apache Solr and Apache Lucene limitations
      • Configuring DSE Search
        • DSE Search reference
          • Search index config
          • Search index schema
          • dsetool search index commands
          • Configuration properties
        • Viewing search index schema and config
        • Customizing the search index schema
        • Changing auto-generated search index settings
        • Using LowerCaseStrField with search indexes
        • Set the location of search indexes
        • DSE Search logging
        • Enabling multi-threaded queries
        • Configuring additional search components
        • Shuffling shards to balance the load
        • Load balancing for distributed search queries
        • Excluding hosts from distributed queries
      • Managing search indexes
        • About search index management
        • Adjusting timeout for index management
        • About search indexes
        • Generating an index with joins disabled
        • Managing search index fields
          • Syntax for changing schema settings
          • Defining index field types
          • Adding a new field type
          • Adding a column to the index
          • Indexing tuples and UDTs fields
            • Tuple configuration example
            • UDT configuration example
            • Nesting tuples and UDTs
            • Tuples and UDTs as CQL map values
          • Indexing map columns
          • Dropping columns from the index
          • Indexing a column for different analysis
        • Configuring search index joins
        • Reloading the search index
        • Removing a search index
        • Updating the index after data expires (TTL)
        • Inserting/updating data
      • Filtering CQL queries with a search index
        • Search index syntax
        • Search index filtering best practices
        • Filtering on terms
          • Filtering on words, phrases, or substrings
          • Advanced term and phrase searches
        • Geospatial queries for Point and LineString
        • Using dynamic fields
        • Joining cores
        • Spatial queries with polygons require JTS
        • Limiting queries by time
        • UDT query examples
        • Querying CQL collections
        • Using date ranges in solr_query
      • Tutorials and demos
        • Creating a healthcare keyspace for tutorials
        • Multi-faceted search using healthcare data
        • Term and phrase searches using the wikipedia demo
          • Using secure cluster
        • Indexing and querying polygons
      • Performance tuning and monitoring DSE Search
        • Tuning search for maximum indexing throughput
        • Resolving query timeouts on restarted nodes
        • Table compression can optimize reads
        • Parallelizing large row reads
        • Changing the stack size and memtable space
        • Tuning index size and range query speed
        • Improving read performance
      • DSE Search operations
        • Initial data migration
        • Shard routing for distributed queries
        • Deleting a search index
        • Verifying indexing status
        • Backing up DSE Search data directories
        • Restoring a search node from backup
        • Monitoring DSE Search
        • Uploading the search index schema and config
      • Solr interfaces
        • Changing the Solr connector port
        • Allowing access from Solr Admin UI for core indexing (deprecated)
        • Changing Tomcat web server settings
        • Configuring the Solr library path
        • Changing the HTTP interface to Apache JServe Protocol
        • URP and FIT
          • FIT transformer API
          • FIT transformer class examples
          • Custom URP example
          • Interface for custom field types
        • Deleting by query
        • Monitoring Solr segments
      • HTTP API SolrJ and other Solr clients
    • DSE Spark
      • About Spark
      • Using Spark with DataStax Enterprise
        • Starting Spark
        • Running Spark commands against a remote cluster
        • Monitoring Spark with the web interface
        • Using DSE Spark with third party tools and integrations
      • Configuring Spark
        • Configuring Spark nodes
        • Automatic Spark Master election
        • Configuring Spark logging options
        • Running Spark processes as separate users
        • Configuring the Spark history server
        • Enabling Spark apps in cluster mode when authentication is enabled
        • Setting Spark Cassandra Connector-specific properties
        • Creating a DSE Analytics Solo datacenter
        • Spark JVMs and memory management
      • Using Spark modules with DataStax Enterprise
        • Getting started with Spark Streaming
        • Using Spark SQL to query data
          • Querying database data using Spark SQL in Scala
          • Querying database data using Spark SQL in Java
          • Querying DSE Graph vertices and edges with Spark SQL
          • Supported syntax of Spark SQL
          • Inserting data into tables with static columns using Spark SQL
          • Running HiveQL queries using Spark SQL
          • Using the DataFrames API
          • Using the Spark SQL Thrift server
          • Enabling SSL for the Spark SQL Thrift Server
          • Accessing the Spark SQL Thrift Server with the Simba JDBC driver
          • Simba ODBC Driver for Apache Spark (Windows)
            • Configuring the Spark ODBC Driver (Windows)
          • Simba ODBC Driver for Apache Spark (Linux)
          • Connecting to the Spark SQL Thrift server using Beeline
        • Using SparkR with DataStax Enterprise
      • Accessing DataStax Enterprise data from external Spark clusters
        • Overview of BYOS support in DataStax Enterprise
        • Generating the BYOS configuration file
        • Connecting to DataStax Enterprise using the Spark shell on an external Spark cluster
        • Generating Spark SQL schema files
        • Starting Spark SQL Thrift Server with Kerberos
        • Accessing HDFS or CFS resources using Kerberos
      • Using the Spark Jobserver
  • DataStax Enterprise 5.1 Documentation
  • DSE Spark
  • Using Spark with DataStax Enterprise
  • Starting Spark

Starting Apache Spark™

Before you start Spark, configure Authorizing remote procedure calls for CQL execution for the DseClientTool object.

RPC permission for the DseClientTool object is required to run Spark because the DseClientTool object is called implicitly by the Spark launcher.

By default DSEFS is required to execute Spark applications. DSEFS should not be disabled when Spark is enabled on a DSE node. If there is a strong reason not to use DSEFS as the default file system, reconfigure Spark to use a different file system. For example, to use a local file system, set the following properties in spark-daemon-defaults.conf:

spark.hadoop.fs.defaultFS=file:///
spark.hadoop.hive.metastore.warehouse.dir=file:///tmp/warehouse
Where is the spark-daemon-defaults.conf file?

The location of the spark-daemon-defaults.conf file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/spark/spark-daemon-defaults.conf

Tarball installations + Installer-No Services installations

<installation_location>/resources/spark/conf/spark-daemon-defaults.conf

How you start Spark depends on the installation and if you want to run in Spark mode or SearchAnalytics mode:

Package and Installer-Services installations

To start the Spark trackers on a cluster of analytics nodes, edit the /etc/default/dse file to set SPARK_ENABLED to 1.

When you start DataStax Enterprise as a service, the node is launched as a Spark node. You can enable additional components.

Mode Option in /etc/default/dse Description

Spark

SPARK_ENABLED=1

Start the node in Spark mode.

SearchAnalytics mode

SPARK_ENABLED=1

SEARCH_ENABLED=1

SearchAnalytics mode requires testing in your environment before it is used in production clusters. In dse.yaml, cql_solr_query_paging: driver is required.

Where is the dse.yaml file?

The location of the dse.yaml file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/dse.yaml

Tarball installations + Installer-No Services installations

<installation_location>/resources/dse/conf/dse.yaml

Tarball and Installer-No Services installations

To start the Spark trackers on a cluster of analytics nodes, use the -k option:

<installation_location>/bin/dse cassandra -k

Nodes started with -k are automatically assigned to the default Analytics datacenter if you do not configure a datacenter in the snitch property file.

You can enable additional components:

Mode Option Description

Spark

-k

Start the node in Spark mode.

SearchAnalytics mode

-k -s

In dse.yaml, cql_solr_query_paging: driver is required.

For example:

To start a node in SearchAnalytics mode, use the -k and -s options.

installation_location/bin/dse cassandra -k -s

Starting the node with the Spark option starts a node that is designated as the master, as shown by the Analytics(SM) workload in the output of the dsetool ring command:

dsetool ring
Address          DC                   Rack         Workload             Graph  Status  State    Load             Owns                 Token                                        Health [0,1]
                                                                                                                                      0
10.200.175.149   Analytics            rack1        Analytics(SM)        no     Up      Normal   185 KiB          ?                    -9223372036854775808                         0.90
10.200.175.148   Analytics            rack1        Analytics(SW)        no     Up      Normal   194.5 KiB        ?                    0                                            0.90
Note: you must specify a keyspace to get ownership information.

Launching Spark

After starting a Spark node, use dse commands to launch Spark.

Table 1. Spark location
Installation Type Location

Package installations + Installer-Services installations

$ dse spark

Tarball installations + Installer-No Services installations

installation_location/bin/dse spark

You can use Cassandra specific properties to start Spark. Spark binds to the listen_address that is specified in cassandra.yaml.

Where is the cassandra.yaml file?

The location of the cassandra.yaml file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/cassandra/cassandra.yaml

Tarball installations + Installer-No Services installations

<installation_location>/resources/cassandra/conf/cassandra.yaml

DataStax Enterprise supports these commands for launching Spark on the DataStax Enterprise command line:

dse spark

Enters interactive Spark shell, offers basic auto-completion.

dse spark-submit

Launches applications on a cluster like spark-submit. Using this interface you can use Spark cluster managers without the need for separate configurations for each application. The syntax for Package and Installer-Services installations is:

dse spark-submit --class class_name jar_file other_options

For example, if you write a class that defines an option named d, enter the command as follows:

dse spark-submit --class com.datastax.HttpSparkStream target/HttpSparkStream.jar -d $NUM_SPARK_NODES

The JAR file can be located in a DSEFS directory. If the DSEFS cluster is secured, provide authentication credentials as described in DSEFS authentication.

The dse spark-submit command supports the same options as Apache Spark’s spark-submit. For example, to submit an application using cluster mode using the supervise option to restart in case of failure:

dse spark-submit --deploy-mode cluster --supervise --class com.datastax.HttpSparkStream target/HttpSparkStream.jar -d $NUM_SPARK_NODES

Unlike the standard behavior for the Spark status and kill options, in DSE deployments these options do not require the Spark Master IP address:

  • dse spark-submit -kill driver_id [--master master_ip_address]

  • dse spark-submit -status driver_id [--master master_ip_address]

For example, to kill a driver of a Spark application running in the DSE cluster:

dse spark-submit --kill driver-20180726160353-0019

To get the status of a Spark application running in the DSE cluster:

dse spark-submit --status driver-20180726160353-0019

The directory in which you run the dse Spark commands must be writable by the current user.

Internal authentication is supported.

Use the optional environment variables DSE_USERNAME and DSE_PASSWORD to increase security and prevent the user name and passwords from appearing in the Spark log files or in the process list on the Spark Web UI. To specify a user name and password using environment variables, add the following to your Bash .profile or .bash_profile:

export DSE_USERNAME=user
export DSE_PASSWORD=secret

These environment variables are supported for all Spark and dse client-tool commands.

DataStax recommends using the environment variables instead of passing user credentials on the command line.

Authentication credentials can be provided in several ways, see Connecting to authentication enabled clusters.

Specifying Spark URLs

You do not need to specify the Spark Master address when starting Spark jobs with DSE. If you connect to any Spark node in a datacenter, DSE automatically discovers the Master address and connects the client to the Master.

Specify the URL for any Spark node using the following format:

dse://[Spark node address[:port number]]?[parameter name=parameter value;]...

By default the URL is dse://?, which is equivalent to dse://localhost:9042. Any parameters you set in the URL override the configuration read from DSE’s Spark configuration settings. Valid parameters are CassandraConnectorConf settings with the spark.cassandra. prefix stripped. For example, you can set the spark.cassandra.connection.local_dc option to dc2 by specifying dse://?connection.local_dc=dc2.

Or to specify multiple spark.cassandra.connection.host addresses for high-availability if the specified connection point is down: dse://1.1.1.1:123?connection.host=1.1.2.2,1.1.3.3.

If the connection.host parameter is specified, the host provided in the standard URL is prepended to the list of hosts set in connection.host. If the port is specified in the standard URL, it overrides the port number set in the connection.port parameter.

Connection options when using dse spark-submit are retrieved in the following order: from the Master URL, then the Spark Cassandra Connector options, then the DSE configuration files.

Detecting Spark application failures

DSE has a failure detector for Spark applications, which detects whether a running Spark application is dead or alive. If the application has failed, the application is removed from the DSE Spark Resource Manager.

The failure detector works by keeping an open TCP connection from a DSE Spark node to the Spark Driver in the application. No data is exchanged, but regular TCP connection keep-alive control messages are sent and received. When the connection is interrupted, the failure detector attempts to reacquire the connection every 1 second for the duration of the appReconnectionTimeoutSeconds timeout value (5 seconds by default). If it fails to reacquire the connection during that time, the application is removed.

A custom timeout value is specified by adding appReconnectionTimeoutSeconds=value in the master URI when submitting the application. For example to set the timeout value to 10 seconds:

dse spark --master dse://?appReconnectionTimeoutSeconds=10
Using Spark with DataStax Enterprise Running Spark commands against a remote cluster

General Inquiries: +1 (650) 389-6000 info@datastax.com

© DataStax | Privacy policy | Terms of use

DataStax, Titan, and TitanDB are registered trademarks of DataStax, Inc. and its subsidiaries in the United States and/or other countries.

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Kubernetes is the registered trademark of the Linux Foundation.

landing_page landingpage