Manage Astra Streaming with the Astra CLI

Use the Astra CLI to manage your Astra Streaming deployments.

Create a tenant

Use the astra streaming create command to create a new tenant:

astra streaming create TENANT_NAME

Result

https://api.astra.datastax.com/v2/streaming/tenants/my-streaming-tenant
[OK]    Tenant 'my-streaming-tenant' has being created.

Running astra streaming create without any options deploys the tenant in the default cloud provider (aws), region (useast2), plan (free) and namespace (default).

All tenant names in Astra Streaming must be unique. If the tenant name is already in use, the command fails with with the following error message:

[ERROR] ALREADY_EXIST: Tenant name 'TENANT_NAME' already exist and must be unique for the cluster.

You can use the --if-not-exist option to only create the tenant if it doesn’t already exist:

astra streaming create TENANT_NAME --if-not-exist

Result

https://api.astra.datastax.com/v2/streaming/tenants/my-streaming-tenant
[OK]    Tenant already existed (--if-not-exist)

Alternatively, you can use the astra streaming exist command to check ahead of time if a tenant name is already in use:

astra streaming exist my-streaming-tenant

Result

https://api.astra.datastax.com/v2/streaming/tenants/my-streaming-tenant
[OK]    Tenant 'my-streaming-tenant' exists.

`streaming create` options

Expand to see all streaming create options

NAME
        astra streaming create - Create a tenant in streaming with cli

SYNOPSIS
        astra streaming create [ {-c | --cloud} <cloudProvider> ]
                [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-cl | --cluster} <cluster> ]
                [ {-conf | --config} <CONFIG_SECTION> ]
                [ {-e | --email} <email> ] [ --env <Environment> ]
                [ {--if-not-exist | --if-not-exists} ] [ --no-color ]
                [ {-o | --output} <FORMAT> ] [ {-p | --plan} <plan> ]
                [ {-r | --region} <cloudRegion> ] [ --token <AUTH_TOKEN> ]
                [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -c <cloudProvider>, --cloud <cloudProvider>
            Cloud Provider to create a tenant

        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -cl <cluster>, --cluster <cluster>
            Dedicated cluster, replacement for cloud/region

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        -e <email>, --email <email>
            User Email

        --env <Environment>
            Astra Environment to use

        --if-not-exist, --if-not-exists
            will create a new DB only if none with same name

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        -p <plan>, --plan <plan>
            Plan for the tenant

        -r <cloudRegion>, --region <cloudRegion>
            Cloud Region for the tenant

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

You can use the -c, -r, and -p options to define the cloud provider, region, and plan when creating a tenant:

astra streaming create TENANT_NAME \
    -c aws \
    -r useast1 \
    -p payg

Result

https://api.astra.datastax.com/v2/streaming/tenants/my-streaming-tenant
[OK]    Tenant 'my-streaming-tenant' has being created.

`streaming exist` options

Expand to see all streaming exist options

NAME
        astra streaming exist - Show existence of a tenant

SYNOPSIS
        astra streaming exist [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

Get tenant details

Use the astra streaming list command to get a list of all the tenants in your organization:

astra streaming list

Result

+---------------------+-----------+----------------+----------------+
| name                | cloud     | region         | status         |
+---------------------+-----------+----------------+----------------+
| my-streaming-tenant | aws       | useast1        | active         |
+---------------------+-----------+----------------+----------------+

Use the astra streaming get or astra streaming describe command to get information about a specific tenant:

astra streaming get TENANT_NAME

astra streaming describe TENANT_NAME

Result

+------------------+-------------------------------------------------------------+
| Attribute        | Value                                                       |
+------------------+-------------------------------------------------------------+
| Name             | my-streaming-tenant                                         |
| status           | active                                                      |
| Cloud Provider   | aws                                                         |
| Cloud region     | useast1                                                     |
| Cluster Name     | pulsar-aws-useast1                                          |
| Pulsar Version   | 2.10.2                                                      |
| Jvm Version      | JDK11                                                       |
| Plan             | payg                                                        |
| WebServiceUrl    | https://pulsar-aws-useast1.api.streaming.datastax.com       |
| BrokerServiceUrl | pulsar+ssl://pulsar-aws-useast1.streaming.datastax.com:6651 |
| WebSocketUrl     | wss://pulsar-aws-useast1.streaming.datastax.com:8001/ws/v2  |
+------------------+-------------------------------------------------------------+

Use the astra streaming status command to get the status of a specific tenant:

astra streaming status TENANT_NAME

Result

[OK]    Tenant 'my-streaming-tenant' has status 'active'

Use the astra streaming pulsar-token command to get the Pulsar token for a specific tenant:

astra streaming pulsar-token TENANT_NAME

Result

eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MzYyMTcxMzAsImlzcyI6ImRhdGFzdGF4Iiwic3ViIjoiY2xpZW50OzJkYmQzYzU1LTZhNjgtNGI1Yi05MTU1LTViZTlkNDE4MjNkODtiWGt0YzNSeVpXRnRhVzVuTFhSbGJtRnVkQT09OzVlMzQ5Y2NlNzgiLCJ0b2tlbmlkIjoiNWUzNDljY2U3OCJ9.aVQiSD7wS21ltrVn-Beei39WC9T9oGGz2P95DjXbHsqB-Lgp2N_T2jTjBxzoKv5-AoCGjMZAK7vr0SvKsyzViXY9ubE72hu0TsnPPFyDcqX-3fpdI0HoA62ppXrqmZvZVKw2bCCg4868xhfGJvY_S5R8K6Zz-2bOuKf8I271V7-gUVw5zlbnkmCER6ch-11Kq3o4HMa9rgoY1W0DNv4V6CNQdjM4qs6qLal0U9Qd3jEKR935jsr518LEye8F4rMGhqNX-Wrnb45kiejrVA4nAdLK6mBMhIZ68lw2J3bQqjCX26NXPsJFmQiR1I0YLFgRJXpoFhYjmA058duTP9Hy4Q

`streaming list` options

Expand to see all streaming list options

NAME
        astra streaming list - Display the list of Tenant in an organization

SYNOPSIS
        astra streaming list [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ]

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

`streaming get` options

Expand to see all streaming get options

NAME
        astra streaming get - Show details of a tenant

SYNOPSIS
        astra streaming get [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ {-k | --key} <Key> ] [ --no-color ]
                [ {-o | --output} <FORMAT> ] [ --token <AUTH_TOKEN> ]
                [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        -k <Key>, --key <Key>
            Show value for a property among: 'status', 'cloud', 'pulsar_token',
            'region'

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

`streaming describe` options

Expand to see all streaming describe options

NAME
        astra streaming describe - Show details of a tenant

SYNOPSIS
        astra streaming describe [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ {-k | --key} <Key> ] [ --no-color ]
                [ {-o | --output} <FORMAT> ] [ --token <AUTH_TOKEN> ]
                [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        -k <Key>, --key <Key>
            Show value for a property among: 'status', 'cloud', 'pulsar_token',
            'region'

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

`streaming status` options

Expand to see all streaming status options

NAME
        astra streaming status - Show status of a tenant

SYNOPSIS
        astra streaming status [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

`streaming pulsar-token` options

Expand to see all streaming pulsar-token options

NAME
        astra streaming pulsar-token - Show status of a tenant

SYNOPSIS
        astra streaming pulsar-token [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

Get a list of available Astra Streaming regions

Use the astra streaming list-regions command to get a list of available cloud provider regions that support Astra Streaming:

astra streaming list-regions

Result

+-----------------+---------------------+-------------------------------+
| Cloud Provider  | Region              | Full Name                     |
+-----------------+---------------------+-------------------------------+
| aws             | apsouth1            |                               |
| aws             | apsoutheast1        |                               |
| aws             | apsoutheast2        |                               |
| aws             | eucentral1          |                               |
| aws             | euwest1             |                               |
| aws             | useast1             |                               |
| aws             | useast2             |                               |
| aws             | uswest2             |                               |
| azure           | australiaeast       |                               |
| azure           | eastus              |                               |
| azure           | westeurope          |                               |
| azure           | westus2             |                               |
| gcp             | australiase1        |                               |
| gcp             | europewest1         |                               |
| gcp             | uscentral1          |                               |
| gcp             | useast1             |                               |
| gcp             | useast4             |                               |
+-----------------+---------------------+-------------------------------+

Supported regions are subject to change and may be different from the sample result.

`streaming list-regions` options

Expand to see all streaming list-regions options

NAME
        astra streaming list-regions - Display the list of Tenant in an
        organization

SYNOPSIS
        astra streaming list-regions [ {-c | --cloud} <CLOUD> ]
                [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ {-f | --filter} <filter> ] [ --no-color ]
                [ {-o | --output} <FORMAT> ] [ --token <AUTH_TOKEN> ]
                [ {-v | --verbose} ]

OPTIONS
        -c <CLOUD>, --cloud <CLOUD>
            Filter on Cloud provider

        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        -f <filter>, --filter <filter>
            Filter on names

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

`streaming list-clouds` options

Expand to see all streaming list-clouds options

NAME
        astra streaming list-clouds - Display the list of clouds

SYNOPSIS
        astra streaming list-clouds [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ]

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

Delete a tenant

Use the astra streaming delete command to delete a tenant:

astra streaming delete TENANT_NAME

Result

[OK]    Deleting Tenant 'my-streaming-tenant'

`streaming delete` options

Expand to see all streaming delete options

NAME
        astra streaming delete - Delete an existing tenant

SYNOPSIS
        astra streaming delete [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

The Pulsar shell is an interactive command-line interface tool for managing and interacting with Pulsar clusters. It is designed to allow developers and administrators to execute Pulsar commands interactively or in a script-like fashion, facilitating tasks such as creating topics, producing and consuming messages, managing schemas, and more.

The Astra CLI provides built-in Pulsar shell support by downloading, installing, configuring, and wapping the pulsar-shell utility.

Use the astra streaming pulsar-shell command to start the Pulsar shell in interactive mode:

astra streaming pulsar-shell TENANT_NAME

Result

[INFO]  Downloading PulsarShell, please wait...
[INFO]  Installing  archive, please wait...
[INFO]  Pulsar-shell is starting please wait for connection establishment...
Using directory: /Users/USERNAME/.pulsar-shell
Welcome to Pulsar shell!
  Service URL: pulsar+ssl://pulsar-aws-useast2.streaming.datastax.com:6651
  Admin URL: https://pulsar-aws-useast2.api.streaming.datastax.com

Type help to get started or try the autocompletion (TAB button).
Type exit or quit to end the shell session.

default(pulsar-aws-useast2.streaming.datastax.com)>

Type exit and press Return to exit the Pulsar shell.

The first time you use the astra streaming pulsar-shell commands, the Astra CLI downloads and installs the pulsar-shell utility to the Astra CLI installation directory (~/.astra/lunastreaming-shell-*).

`streaming pulsar-shell` options

Expand to see all streaming pulsar-shell options

NAME
        astra streaming pulsar-shell - Start pulsar admin against your tenant

SYNOPSIS
        astra streaming pulsar-shell [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ]
                [ {-e | --execute-command} <command> ] [ --env <Environment> ]
                [ {-f | --filename} <FILE> ] [ --fail-on-error ] [ --no-color ]
                [ {-np | --no-progress} ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        -e <command>, --execute-command <command>
            Execute the statement and quit.

        --env <Environment>
            Astra Environment to use

        -f <FILE>, --filename <FILE>
            Input filename with a list of commands to be executed. Each command
            must be separated by a newline.

        --fail-on-error
            If true, the shell will be interrupted if a command throws an
            exception.

        --no-color
            Remove all colors in output

        -np, --no-progress
            Display raw output of the commands without the fancy progress
            visualization.

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant unique name

Use the -e/--execute option to execute a Pulsar shell command:

astra streaming pulsar-shell TENANT_NAME -e "admin namespaces list TENANT_NAME"

Use the -f/--file option to execute Pulsar shell commands from a file:

astra streaming pulsar-shell TENANT_NAME -f sample.txt

Pulsar shell example

The following procedure shows an example of how to use the Astra CLI’s built-in Pulsar shell support to write and read in a topic with a client:

Create a tenant:

astra streaming create my-streaming-tenant

Result

https://api.astra.datastax.com/v2/streaming/tenants/my-streaming-tenant
[OK]    Tenant 'my-streaming-tenant' has being created.

In two separate terminals, run the following command to start the Pulsar shell in each terminal:

astra streaming pulsar-shell my-streaming-tenant

Result

[INFO]  Pulsar-shell is starting please wait for connection establishment...
Using directory: /Users/USERNAME/.pulsar-shell
Welcome to Pulsar shell!
  Service URL: pulsar+ssl://pulsar-aws-useast2.streaming.datastax.com:6651
  Admin URL: https://pulsar-aws-useast2.api.streaming.datastax.com

Type help to get started or try the autocompletion (TAB button).
Type exit or quit to end the shell session.

In the first terminal, create a topic named demo in the default namespace:
```
admin topics create persistent://my-streaming-tenant/default/demo
```

Confirm that the topic was successfully created in the default namespace:

admin topics list my-streaming-tenant/default

Result

persistent://my-streaming-tenant/default/demo

Start a consumer on the demo topic:

client consume persistent://my-streaming-tenant/default/demo -s astra_cli_tuto -n 0

Result

.. init ...
83 - R:pulsar-aws-useast2.streaming.datastax.com/3.16.119.226:6651]] Connected to server
2022-09-12T12:28:34,869+0200 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xc5ce3ec4, L:/192.168.82.1:53683 - R:pulsar-aws-useast2.streaming.datastax.com/3.16.119.226:6651] Connected through proxy to target broker at 192.168.7.141:6650
2022-09-12T12:28:35,460+0200 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-streaming-tenant/default/demo/default/demo][astra_cli_tuto] Subscribing to topic on cnx [id: 0xc5ce3ec4, L:/192.168.82.1:53683 - R:pulsar-aws-useast2.streaming.datastax.com/3.16.119.226:6651], consumerId 0
2022-09-12T12:28:35,645+0200 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-streaming-tenant/default/demo/default/demo][astra_cli_tuto] Subscribed to topic on pulsar-aws-useast2.streaming.datastax.com/3.16.119.226:6651 -- consumer: 0

In the second terminal, start a producer.

client produce persistent://my-streaming-tenant/default/demo/default/demo -m "hello world" -n 20

Result

2022-09-12T12:36:28,684+0200 [pulsar-client-io-14-1] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0x682890b5, L:/192.168.1.106:53796 ! R:pulsar-aws-useast2.streaming.datastax.com/3.138.177.230:6651] Disconnected
2022-09-12T12:36:30,756+0200 [main] INFO  org.apache.pulsar.client.cli.PulsarClientTool - 40 messages successfully produced


And on the client side
key:[null], properties:[], content:world
----- got message -----
key:[null], properties:[], content:hello

Change Data Capture (CDC)

CDC for Astra DB automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into Astra Streaming where it can be processed by client applications or sent to downstream systems.

For more information about using the Astra CLI to configure CDC, see Configure Change Data Capture.

Use the streaming list-cdc command to list the CDC connections associated with a tenant:

astra streaming list-cdc TENANT_NAME

Result

+--------------------+----------------+----------------+-------------------+----------------+----------------+
| cluster            | namespace      | database       | keyspace          | table          | status         |
+--------------------+----------------+----------------+-------------------+----------------+----------------+
| pulsar-aws-useast1 | astracdc       | cdc_demo_db    | cdc_demo_keyspace | cdc_demo_table | running        |
+--------------------+----------------+----------------+-------------------+----------------+----------------+

`streaming list-cdc` options

Expand to see all streaming list-cdc options

NAME
        astra streaming list-cdc - List CDC available on this tenant

SYNOPSIS
        astra streaming list-cdc [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

Create environment variables

When developing applications with Astra Streaming APIs, you will notice common values that are frequently reused, such as Pulsar tokens, broker URLs, and web service URLs. It’s a good practice to store these values in environment variables to facilitate reuse in your scripts, simplify token rotation, and increase security by keeping sensitive information separate from your core application code.

The Astra CLI can automatically generate a set of relevant variables for an Astra Streaming tenant that you can use in your applications.

Use the astra streaming create-dotenv command to generate a .env file containing relevant environment variables for a specific Astra Streaming tenant:

astra streaming create-dotenv TENANT_NAME

By default, the Astra CLI saves the .env file to the directory where you run the command.

If a .env file already exists in the directory, the Astra CLI appends the new environment variables to the existing file.

Result

.env

ASTRA_STREAMING_BROKER_URL="pulsar+ssl://pulsar-aws-useast2.streaming.datastax.com:6651"
ASTRA_STREAMING_CLOUD="aws"
ASTRA_STREAMING_NAME="my-streaming-tenant"
ASTRA_STREAMING_PULSAR_TOKEN="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."
ASTRA_STREAMING_REGION="useast2"
ASTRA_STREAMING_WEBSERVICE_URL="https://pulsar-aws-useast2.api.streaming.datastax.com"
ASTRA_STREAMING_WEBSOCKET_URL="wss://pulsar-aws-useast2.streaming.datastax.com:8001/ws/v2"

The generated .env files contains sensitive information, including the application token that the Astra CLI uses to connect to Astra.

The SCB contains sensitive information that establishes a connection to your database, including key pairs and certificates.

`streaming create-dotenv` options

Expand to see all streaming create-dotenv options

NAME
        astra streaming create-dotenv - Generate an .env configuration file
        associate with the tenant

SYNOPSIS
        astra streaming create-dotenv [ {-cf | --config-file} <CONFIG_FILE> ]
                [ {-conf | --config} <CONFIG_SECTION> ]
                [ {-d | --directory} <DIRECTORY> ] [ --env <Environment> ]
                [ --no-color ] [ {-o | --output} <FORMAT> ]
                [ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>

OPTIONS
        -cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
            Configuration file (default = ~/.astrarc)

        -conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
            Section in configuration file (default = ~/.astrarc)

        -d <DIRECTORY>, --directory <DIRECTORY>
            Destination for the config file

        --env <Environment>
            Astra Environment to use

        --no-color
            Remove all colors in output

        -o <FORMAT>, --output <FORMAT>
            Output format, valid values are: human,json,csv

        --token <AUTH_TOKEN>
            Key to use authenticate each call.

        -v, --verbose
            Verbose mode with log in console

        --
            This option can be used to separate command-line options from the
            list of arguments (useful when arguments might be mistaken for
            command-line options)

        <TENANT>
            Tenant identifier

Use the -d/--directory option to save the .env file to a specified directory:

astra streaming create-dotenv TENANT_NAME -d ~/tmp

Result

[OK]    File '/Users/USERNAME/tmp/.env' has been created/amended

The directory must already exist before you run the command. If the directory doesn’t exist, the Astra CLI will report the following error: INVALID_ARGUMENT: Destination folder has not been found.

Manage Astra Streaming with the Astra CLI

Create a tenant

`streaming create` options

`streaming exist` options

Get tenant details

`streaming list` options

`streaming get` options

`streaming describe` options

`streaming status` options

`streaming pulsar-token` options

Get a list of available Astra Streaming regions

`streaming list-regions` options

`streaming list-clouds` options

Delete a tenant

`streaming delete` options

Use the Pulsar shell

`streaming pulsar-shell` options

Pulsar shell example

Change Data Capture (CDC)

`streaming list-cdc` options

Create environment variables

`streaming create-dotenv` options

Was this helpful?

Give Feedback