Manage Apache Spark application permissions

Manage user access to Apache Spark™ applications. The CQL resources for Spark applications are WORKPOOL and SUBMISSION. Create permissions on the workpool resource controls the ability of a user to submit a Spark application to DSE. Modify permissions on submission resource controls the ability of a user to manage and remove applications.

Procedure

Use CQL shell (cqlsh) to authorize access to DSE Resource Manager and Spark applications. All commands must be entered on a DSE Analytics node in the cluster.

Access required for AlwaysOn SQL roles:

GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlRoutingRPC to <role_name>;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseResourceManager TO <role_name>;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseClientTool TO <role_name>;
GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_cache_table TO <role_name>;
GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_info TO <role_name>;

Access to only DSE Resource Manager:

GRANT EXECUTE ON REMOTE OBJECT DseResourceManager TO <role_name>;

Run applications:
```
GRANT EXECUTE ON REMOTE OBJECT DseClientTool TO <role_name>
```
Each DSE Analytics user must have permission to make remote procedure calls with DSE client tools.

For roles that are not superusers, access to the following tables is required:

GRANT SELECT ON system.size_estimates TO <role_name>;
GRANT SELECT, MODIFY ON "HiveMetaStore".sparkmetastore TO <role_name>;

Additional permissions are required when running AlwaysOn SQL:

GRANT SELECT, MODIFY ON dse_analytics.alwayson_sql_cache_table TO <role_name>;

Submit applications:
- To all datacenters:
  GRANT CREATE ON ANY WORKPOOL TO <role_name>;
  Use revoke command to remove access:
  
  REVOKE CREATE ON ANY WORKPOOL FROM <role_name>;
- A particular datacenter:
  GRANT CREATE ON WORKPOOL <datacenter_name> TO <role_name>;
  Use revoke command to remove access:
  
  REVOKE CREATE ON WORKPOOL <datacenter_name> FROM <role_name>;
  The role used to submit an application is automatically granted permission to MODIFY the application.

Modify applications:

All applications:

GRANT MODIFY ON ANY SUBMISSION TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON ANY SUBMISSION FROM <role_name>;

All applications in a particular datacenter:

GRANT MODIFY ON ANY SUBMISSION IN WORKPOOL '<datacenter_name>.*' TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON ANY SUBMISSION IN WORKPOOL '<datacenter_name>.*' FROM <role_name>;

You must specify a workpool name or wildcard when specifying a datacenter. In DSE versions prior to 6.0, you could specify the datacenter name only, but omitting the workpool name or wildcard results in a syntax error.

Specific application in a particular datacenter:

GRANT MODIFY ON SUBMISSION <id> IN WORKPOOL '<datacenter_name>.*' TO <role_name>;

Use revoke command to remove access:

REVOKE MODIFY ON SUBMISSION <id> IN WORKPOOL '<datacenter_name>.*' FROM <role_name>;

For example, to revoke the MODIFY permissions of an application started by a role named sparkrole, first find the application ID:

LIST ALL PERMISSIONS OF sparkrole;

role      | username  | resource
----------+-----------+-----------------------------------------------------------
...
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
sparkrole | sparkrole | <submission app-20190519161729-0004 in work pool default in Analytics>
...

| permission | granted | restricted | grantable
+------------+---------+------------+-----------
...
|     MODIFY |    True |      False |     False
|  AUTHORIZE |    True |      False |     False
|   DESCRIBE |    True |      False |     False
...

In the resource column the application ID follows submission. Here, the application ID is app-20190519161729-0004.

Use the REVOKE command, specifying the role name, datacenter, and workpool listed in the resource column in the previous command.
```
REVOKE MODIFY ON SUBMISSION 'app-20190519161729-0004' IN WORKPOOL 'Analytics.default' FROM sparkrole;
```

To revoke all permissions from the application, use REVOKE ALL:

REVOKE ALL ON SUBMISSION 'app-20190519161729-0004' IN WORKPOOL 'Analytics.default' FROM sparkrole;

GRANT EXECUTE ON REMOTE OBJECT DseGraphRpc TO <role_name>;

View worker logs:

One of the following three permissions are required to view worker logs:
- GRANT DESCRIBE ON ANY SUBMISSION TO cassdev;
- GRANT DESCRIBE ON ANY SUBMISSION IN WORKPOOL 'datacenter_name.*' TO role_name;
- GRANT DESCRIBE ON SUBMISSION id IN WORKPOOL 'datacenter_name.*' TO role_name;

Example

Create role for AlwaysOn SQL (alwayson_sql):

CREATE ROLE alwayson_sql WITH LOGIN=true; // role name matches auth_user

// Required if scheme_permissions true
GRANT EXECUTE ON ALL AUTHENTICATION SCHEMES TO alwayson_sql;

// {spark-short} RPC settings
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseResourceManager TO alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT DseClientTool TO alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlRoutingRPC to alwayson_sql;
GRANT ALL PERMISSIONS ON REMOTE OBJECT AlwaysOnSqlNonRoutingRPC to alwayson_sql;

// {spark} and {product-short} required table access
GRANT SELECT ON system.size_estimates TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON "HiveMetaStore".sparkmetastore TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON dse_analytics.alwayson_sql_cache_table TO alwayson_sql;
GRANT SELECT, UPDATE, TRUNCATE ON dse_analytics.alwayson_sql_info TO alwayson_sql;

// Permissions to create and change applications
GRANT CREATE, DESCRIBE ON ANY WORKPOOL TO alwayson_sql;
GRANT UPDATE, TRUNCATE, DESCRIBE ON ANY SUBMISSION TO alwayson_sql;

Manage Apache Spark application permissions

Procedure

Example

Was this helpful?

Give Feedback