Setting up DSE Spark Application Permissions

Manage user access to Spark applications. The CQL resources for Spark applications are WORKPOOL and SUBMISSION. Create permissions on the workpool resource controls the ability of a user to submit a Spark application to DSE. Modify permissions on submission resource controls the ability of a user to manage and remove applications.

Procedure

Use CQL shell (cqlsh) to authorize access to DSE Resource Manager and Spark applications. All commands must be entered on a DSE Analytics node in the cluster.

Access required for AlwaysOn SQL roles:

GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlRoutingRPC to <role_name>;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseResourceManager TO <role_name>;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseClientTool TO <role_name>;
GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_cache_table TO <role_name>;
GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_info TO <role_name>;

Access to only DSE Resource Manager:

GRANT EXECUTE ON REMOTE OBJECT DseResourceManager TO <role_name>;

Run applications:
```
GRANT EXECUTE ON REMOTE OBJECT DseClientTool TO <role_name>
```
Each DSE Analytics user must have permission to make remote procedure calls with DSE client tools.

For roles that are not superusers, access to the following tables is required:

GRANT SELECT ON system.size_estimates TO <role_name>;
GRANT SELECT, MODIFY ON "HiveMetaStore".sparkmetastore TO <role_name>;

Additional permissions are required when running AlwaysOn SQL:

GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_cache_table TO <role_name>;

Submit applications:
- To all datacenters:
  GRANT CREATE ON ANY WORKPOOL TO <role_name>;
  Use revoke command to remove access:
  REVOKE CREATE ON ANY WORKPOOL FROM <role_name>;
- A particular datacenter:
  GRANT CREATE ON WORKPOOL <datacenter_name> TO <role_name>;
  Use revoke command to remove access:
  REVOKE CREATE ON WORKPOOL <datacenter_name> FROM <role_name>;
The role used to submit an application is automatically granted permission to MODIFY the application.

Modify applications:

All applications:

GRANT MODIFY ON ANY SUBMISSION TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON ANY SUBMISSION FROM <role_name>;

All applications in a particular datacenter:

GRANT MODIFY ON ANY SUBMISSION IN WORKPOOL '<datacenter_name>.*' TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON ANY SUBMISSION IN WORKPOOL '<datacenter_name>.*' FROM <role_name>;

You must specify a workpool name or wildcard when specifying a datacenter. In DSE versions prior to 6.0, you could specify the datacenter name only, but omitting the workpool name or wildcard results in a syntax error.

Specific application in a particular datacenter:

GRANT MODIFY ON SUBMISSION <id> IN WORKPOOL '<datacenter_name>.*' TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON SUBMISSION <id> IN WORKPOOL '<datacenter_name>.*' FROM <role_name>;

For example, to revoke the MODIFY permissions of an application started by a role named sparkrole, first find the application ID:

LIST ALL PERMISSIONS OF sparkrole;

role      | username  | resource
----------+-----------+-----------------------------------------------------------
...
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
...

| permission | granted | restricted | grantable
+------------+---------+------------+-----------
...
|     MODIFY |    True |      False |     False
|  AUTHORIZE |    True |      False |     False
|   DESCRIBE |    True |      False |     False
...

In the resource column the application ID follows submission. Here, the application ID is app-20190519161729-0004.

Use the REVOKE command, specifying the role name, datacenter, and workpool listed in the resource column in the previous command.

REVOKE MODIFY ON SUBMISSION 'app-20190519161729-0004' IN WORKPOOL 'Analytics.default' FROM sparkrole;

To revoke all permissions from the application, use REVOKE ALL:

REVOKE ALL ON SUBMISSION 'app-20190519161729-0004' IN WORKPOOL 'Analytics.default' FROM sparkrole;

GRANT EXECUTE ON REMOTE OBJECT DseGraphRpc TO <role_name>;

Example

Create role for AlwaysOn SQL (alwayson_sql):

CREATE ROLE alwayson_sql WITH LOGIN=true; // role name matches auth_user

// Required if scheme_permissions true
GRANT EXECUTE ON ALL AUTHENTICATION SCHEMES TO alwayson_sql;

// Spark RPC settings
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseResourceManager TO alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseClientTool TO alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlRoutingRPC to alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlNonRoutingRPC to alwayson_sql;

// Spark and DSE required table access
GRANT SELECT ON system.size_estimates TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON "HiveMetaStore".sparkmetastore TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON dse_analytics.alwayson_sql_cache_table TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON dse_analytics.alwayson_sql_info TO alwayson_sql;

// Permissions to create and change applications
GRANT CREATE, DESCRIBE ON ANY WORKPOOL TO alwayson_sql;
GRANT UPDATE, TRUNCATE, DESCRIBE ON ANY SUBMISSION TO alwayson_sql;