Use the Astra Streaming DevOps API
The Astra Streaming DevOps API is one of the APIs that you can use to manage your Apache Pulsar™ tenants and their resources. Use this API to manage higher-level Astra Streaming objects associated with your Astra organization, such as the Change Data Capture (CDC) settings, Pulsar tenants, georeplication, Pulsar stats, and Pulsar tokens.
|
More endpoints
This page summarizes commonly used endpoints. For more information and all endpoints, see the Astra Streaming DevOps API specification reference. |
Other available APIs include the Astra Streaming Pulsar Admin API and the OSS Apache Pulsar™ REST APIs.
Prerequisites
To use the Astra Streaming DevOps API, you need the following:
-
An active Astra account with access to Astra Streaming.
-
An Astra application token that has a role with Astra Streaming permissions.
-
Depending on the operations you need to perform, you might need additional information about your tenant, such as the cluster name or Pulsar instance name. You can get this information with
GET "https://api.astra.datastax.com/v2/streaming/tenants"or from the Tenant Details section in the Astra Portal.
Set environment variables
Due to their frequency in Astra Streaming DevOps API calls, you might find it helpful to set environment variables for the credentials and tenant details:
export INSTANCE="PULSAR_INSTANCE_NAME"
export TENANT="TENANT_NAME"
export NAMESPACE="NAMESPACE_NAME"
export TOPIC="TOPIC_NAME"
export ASTRA_TOKEN="ASTRA_APPLICATION_TOKEN"
To learn how to get these values, see Prerequisites.
The examples in this guide use environment variables for these values.
For example, --header "Authorization: Bearer $ASTRA_TOKEN".
Format responses
The default response is a single JSON string.
You can use modifications like | jq . or | python3 -mjson.tool to format the output for easier reading.
Formatting the response is optional; it isn’t required to execute API requests.
Many examples in this reference use | python3 -mjson.tool to format the JSON response.
Additionally, the examples use the -sS --fail options to suppress trivial output and handle errors more gracefully while passing the content of successful responses to stdout.
Omit the -sS --fail options for debugging or to get the full output, including progress bars and warnings.
Tenant operations
Use the following Astra Streaming DevOps API endpoints to manage Astra Streaming Pulsar tenants.
Get tenant details
curl -sS --fail -L -X GET "https://api.astra.datastax.com/v2/streaming/tenants" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
| python3 -mjson.tool
Result
[
{
"id": "14b77c47-bdfd-4ba1...",
"tenantName": "mytenant",
"clusterName": "pulsar-aws-useast2",
"webServiceUrl": "https://pulsar-aws-useast2",
"brokerServiceUrl": "pulsar+ssl://pulsar-aws-useast2:6651",
"websocketUrl": "wss://pulsar-aws-useast2:8001/ws/v2",
"websocketQueryParamUrl": "wss://pulsar-aws-useast2:8964/ws/v2",
"pulsarToken": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpX...",
"plan": "payg",
"planCode": "1",
"astraOrgGUID": "b282a256-b129-...",
"cloudProvider": "aws",
"cloudProviderCode": "1",
"cloudRegion": "useast2",
"status": "active",
"jvmVersion": "JDK11",
"pulsarVersion": "2.10.2",
"regionZone": "na",
"Email": "",
"userMetricsUrl": "https://prometheus-aws-useast2...",
"pulsarInstance": "prod0"
},
{
"id": "e8bf25d8-a6a1-4169-...",
"tenantName": "mytenant2",
"clusterName": "pulsar-gcp-useast1",
"webServiceUrl": "https://pulsar-gcp-useast1",
"brokerServiceUrl": "pulsar+ssl://pulsar-gcp-useast1:6651",
"websocketUrl": "wss://pulsar-gcp-useast1m:8001/ws/v2",
"websocketQueryParamUrl": "wss://pulsar-gcp-useast1:8964/ws/v2",
"pulsarToken": "eyJhbGciOiJSUz...",
"plan": "payg",
"planCode": "1",
"astraOrgGUID": "b282a256-b129-43e9...",
"cloudProvider": "gcp",
"cloudProviderCode": "2",
"cloudRegion": "useast1",
"status": "active",
"jvmVersion": "JDK11",
"pulsarVersion": "2.10.2",
"regionZone": "na",
"Email": "",
"userMetricsUrl": "https://prometheus-gcp-useast1...",
"pulsarInstance": "prod0"
}
]
Get cloud providers and regions
Get cloud provider and regions supported by Astra Streaming:
curl -sS --fail -L -X GET "https://api.astra.datastax.com/v2/streaming/providers" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
| python3 -mjson.tool
Result
{
"aws": [
"useast1",
"uswest2",
"useast2"
],
"azure": [
"westus2",
"eastus",
"australiaeast"
],
"gcp": [
"useast1",
"uscentral1",
"australiase1",
"europewest1",
"useast4"
]
}
Create a tenant
To create a tenant, you must provide the required tenant configuration settings in JSON format. You can do this directly in the request body or with a JSON file.
-
Create a tenant using in-line input:
curl -sS --fail -L -X POST "https://api.astra.datastax.com/v2/streaming/tenants" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer $ASTRA_TOKEN" \ --data-raw '{ "cloudProvider": "aws", "cloudRegion": "useast2", "tenantName": "mytenant", "userEmail": "joshua@example.com" }' \ | python3 -mjson.tool -
Create a tenant using file input:
curl -sS --fail -L -X POST "https://api.astra.datastax.com/v2/streaming/tenants" \ --header "Content-Type: application/json" \ --header "Authorization: Bearer $ASTRA_TOKEN" \ --data "@mytenant-config.json" \ | python3 -mjson.toolTo use a file as input for a curl command, use the
@symbol followed by the file path and name.Relative paths are resolved from the current working directory.
The file must be in a location that the shell can access, and the user running the command must have permission to read the file.
The output includes the new tenant’s details and a pulsarToken, which is an initial Pulsar JWT for the new Pulsar instance:
{
"namespace": "default",
"topic": "",
"id": "",
"tenantName": "mytenant",
"clusterName": "pulsar-aws-useast2",
"webServiceUrl": "https://pulsar-aws-useast2",
"brokerServiceUrl": "pulsar+ssl://pulsar-aws-useast2:6651",
"websocketUrl": "wss://pulsar-aws-useast2:8001/ws/v2",
"websocketQueryParamUrl": "wss://pulsar-aws-useast2:8964/ws/v2",
"pulsarToken": "eyJ...",
"plan": "payg",
"planCode": "",
"astraOrgGUID": "b282a256-b129-43e9...",
"cloudProvider": "aws",
"cloudProviderCode": "",
"cloudRegion": "useast2",
"status": "active",
"jvmVersion": "JDK11",
"pulsarVersion": "2.10.2",
"regionZone": "",
"Email": "",
"userMetricsUrl": "",
"pulsarInstance": ""
}
Delete a tenant
curl -sS --fail -L -X DELETE "https://api.astra.datastax.com/v2/streaming/tenants/$TENANT/clusters/$CLUSTER" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $ASTRA_TOKEN"
Georeplication operations
Use the following Astra Streaming DevOps API endpoints to manage georeplication for Astra Streaming namespaces. For more information, see Georeplication.
Get georeplication settings
Get the georeplication settings for a namespace:
curl -sS --fail -L -X GET "https://api.astra.datastax.com/v2/streaming/replications/$INSTANCE/$TENANT/$NAMESPACE" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
| python3 -mjson.tool
If georeplication is enabled, the response includes the replication configuration details.
Result
{
"pulsarInstance": "prod0",
"tenant": "mytenant",
"namespace": "mynamespace",
"replications": {
"pulsar-aws-useast2": [
"pulsar-aws-uswest2",
"pulsar-aws-useast2"
],
"pulsar-aws-uswest2": [
"pulsar-aws-uswest2",
"pulsar-aws-useast2"
]
},
"clusters": {
"pulsar-aws-useast2": {
"clusterName": "pulsar-aws-useast2",
"cloudProvider": "aws",
"cloudRegion": "useast2",
"clusterType": "cloud",
"webServiceUrl": "https://pvt-pulsar-aws-useast2:8443",
"brokerServiceUrl": "pulsar+ssl://pulsar-aws-useast2:6651",
"websocketUrl": "",
"pulsarInstance": "prod0",
"regionZone": ""
},
"pulsar-aws-uswest2": {
"clusterName": "pulsar-aws-uswest2",
"cloudProvider": "aws",
"cloudRegion": "uswest2",
"clusterType": "cloud",
"webServiceUrl": "https://pvt-pulsar-aws-uswest2:8443",
"brokerServiceUrl": "pulsar+ssl://pulsar-aws-uswest2:6651",
"websocketUrl": "",
"pulsarInstance": "prod0",
"regionZone": ""
}
...TRUNCATED FOR READABILITY...
}
}
Enable georeplication
To enable georeplication, provide the origin and destination clusters, the namespace, bidirectional flag, and the email address (if set for the tenant):
curl -sS --fail -L -X POST "https://api.astra.datastax.com/v2/streaming/replications/$INSTANCE/$TENANT/$NAMESPACE" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--data-raw '{
"bidirection": true,
"destCluster": "pulsar-aws-uswest2",
"email": "",
"namespace": "ns0",
"originCluster": "pulsar-aws-useast2"
}'
You can get the clusters, email, and namespace from Get tenant details.
Disable (remove) georeplication
Specify the replication to remove by providing the entire georeplication settings object, including the origin and destination clusters, the namespace, bidirectional flag, and email address (if set for the tenant):
curl -sS --fail -L -X DELETE "https://api.astra.datastax.com/v2/streaming/replications/$INSTANCE/$TENANT/$NAMESPACE" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--data-raw '{
"bidirection": true,
"destCluster": "pulsar-aws-uswest2",
"email": "",
"namespace": "ns0",
"originCluster": "pulsar-aws-useast2"
}'
You can get the clusters, email, and namespace from Get tenant details.
Pulsar token (JWT) operations
Use the following Astra Streaming DevOps API endpoints to manage Pulsar tokens. For more information, see Manage tokens.
|
For Astra application tokens, use the core Astra DevOps API. |
Get existing Pulsar tokens
Get a list of token IDs for a given tenant and cluster:
curl -sS --fail -L -X GET "https://api.astra.datastax.com/v2/streaming/tenants/$TENANT/tokens" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--header "X-DataStax-Pulsar-Cluster: $CLUSTER" \
| python3 -mjson.tool
You can get the cluster name from Get tenant details.
The result is a list of token objects, each containing the token ID and other metadata. You can use the token ID to get the corresponding Pulsar token (JWT) string.
[
{
"iat": 1679335276,
"iss": "datastax",
"sub": "client;b282a256-b129-43e9-b870...",
"tokenid": "cdb87797..."
}
]
Token IDs are also listed in the Astra Portal for each tenant and cluster.
Get the token string for a Pulsar token ID
Get the raw string JWT associated with a specific token ID and cluster:
curl -sS --fail -L -X GET "https://api.astra.datastax.com/v2/streaming/tenants/$TENANT/tokens/PULSAR_TOKEN_ID" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--header "X-DataStax-Pulsar-Cluster: $CLUSTER"
You can get token IDs from Get existing Pulsar tokens, and you can get the cluster name from Get tenant details.
Result
Output: Raw string JWT
eyJhbGciOiJSUzI1NiIsI...
Create a Pulsar token
Create a new Pulsar token (JWT) for a given tenant and cluster. You can get the cluster name from Get tenant details.
-
v2 endpoint
-
v3 endpoint
The v2/streaming/tenants/$TENANT/tokens endpoint returns a raw string JWT with an unlimited lifetime:
curl -sS --fail -L -X POST "https://api.astra.datastax.com/v2/streaming/tenants/$TENANT/tokens" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--header "X-DataStax-Pulsar-Cluster: $CLUSTER"
Result
Output: new raw string JWT
eyJhbGciOiJSUzI1NiIsI...
The v3/streaming/tenants/$TENANT/tokens endpoint returns a JSON response and accepts additional token parameters:
curl -sS --fail -L -X POST "https://api.astra.datastax.com/v3/streaming/tenants/$TENANT/tokens" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--header "X-DataStax-Pulsar-Cluster: $CLUSTER" \
--data-raw '{
"exp": "EXPIRATION",
"role": "ROLE",
"type": "TYPE"
}'
Replace the following:
-
EXPIRATION: The token lifetime expressed as a number and unit, such as1dfor one day. -
ROLE: A string describing the token’s role. -
TYPE: The token’s privilege level, eitheradminornon-admin.
Result
{
"id": "cdb87797...",
"rolename": "admin",
"token": "eyJhbGciOiJSUzI1NiIsI..."
}
Delete a Pulsar token
Delete a token for a given cluster by token ID:
curl -sS --fail -L -X DELETE "https://api.astra.datastax.com/v2/streaming/tenants/$TENANT/tokens/PULSAR_TOKEN_ID" \
--header "Authorization: Bearer $ASTRA_TOKEN" \
--header "X-DataStax-Pulsar-Cluster: $CLUSTER"
You can get the token ID from Get existing Pulsar tokens, and you can get the cluster name from Get tenant details.