Starting Apache Spark™ SQL Thrift Server with Kerberos

Spark SQL Thrift Server is a long running service and must be configured to start with a keytab file if Kerberos is enabled. The user principal must be added to DSE, and Spark SQL Thrift Server restarted with the generated BYOS configuration file and byos-version.jar.

Prerequisites

These instructions are for the Spark SQL Thrift Server included in HortonWorks 2.4. The Hadoop Spark SQL Thrift Server principal is hive/_HOST@REALM.

Procedure

  1. Create the principal on the DSE node using cqlsh.

    cqlsh> create user hive/spark_sql_thrift_server_host@REALM;
  2. Login as the hive user on the Spark SQL Thrift Server host.

  3. Create a ~/.java.login.config file with a JAAS Kerberos configuration.

  4. Merge the existing Spark SQL Thrift Server configuration properties with the generated BYOS configuration file into a new file.

    cat /usr/hdp/current/spark-thriftserver/conf/spark-thrift-sparkconf.conf byos.properties > custom-sparkconf.conf
  5. Start Spark SQL Thrift Server with the custom configuration file and byos-version.jar.

    /usr/hdp/2.4.2.0-258/spark/sbin/start-thriftserver.sh --jars byos-version.jar --properties-file custom-sparkconf.conf
  6. Connect using the Beeline client.

    beeline -u 'jdbc:hive2://hostname:port/default;principal=hive/_HOST@REALM'

Next steps

Generated SQL schema files can be passed to beeline with the -f option to generate a mapping for DSE tables so both Hadoop and DataStax Enterprise tables are available through the service for queries.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com