Astra Streaming offers two types of serverless environments: Pay As You Go and dedicated. If your Astra organization is on the Free plan, you use the Pay As You Go streaming option. When you provide a payment method, you only pay for resources used when messages are produced and consumed. Therefore, you pay as you go. In Pay As You Go streaming environments, your data and interaction with Pulsar are secured over a (public) internet connection, and there are limitations to how many resources you can create.

A dedicated environment is your own private Pulsar cluster with the additional benefits of Astra Streaming. Sign in to Astra just as you would with a Pay As You Go streaming account. When you create new tenants, additional options are available for deploying to your private cluster(s). There are less limits in dedicated environments than in Pay As You Go - it’s your cluster, after all. Finally, billing for a dedicated cluster is unique to each customer.
Contact the team to learn more.

Running a production grade Pulsar cluster that can handle at-scale workloads is not a trivial task. It requires many (virtual) machines to be configured in a very particular way.

In traditional cloud environments, you would pay hourly for every machine whether they are being used for workloads or not, and you would carry the burden of maintaining the server infrastructure.

Serverless removes those operational burdens, and you pay only for the resources you actually use.

All connections in Astra Streaming are guarded by AuthN, AuthZ, and secure (TLS) communications. With a dedicated cluster you have the option to connect over the (public) internet or establish a private connection. To learn more about private connections, refer to your cloud provider’s private link documentation:

Pay As You Go streaming accounts use a shared Pulsar cluster. Without dedicated cloud resources, a private link typically can’t be established. Email the team if you would like to explore this option.

As a Pay As You Go customer, the Astra platform offers single sign-on through your GitHub account and your Google account. Astra also offers custom SSO integration as a premium option. Email the team for more information.

Pulsar has the concept of clients with role tokens. Authentication in Pulsar is the process of verifying a provided (JWT) token, and authorization is the process of determining if the role claimed in that token is allowed to complete the requested action.

Astra Streaming uses the DataStax version of Apache Pulsar (called Luna Streaming). The Luna project is an open fork of the Pulsar project that maintains feature parity with OSS Pulsar. Astra Streaming, as a managed service, abstracts some features/options of Pulsar to ensure continuous, reliable service.

Your Pay As You Go environment is an Astra Organization (Org) that has a tenant (or multiple tenants) on a shared Pulsar cluster. Each of your tenants is secured by Pulsar AuthN & AuthZ models and the Astra Org AuthN and AuthZ. The shared cluster is created and administered by Astra Streaming Admins. Each tenant is assigned a custom role (and permission) limited to only that tenant. All tokens created within a tenant are assigned roles similar to the assigning tenant.

The Astra platform offers different layers of authentication based on the desired action. In general, actions related to your Astra Org (members, org billing, usage metrics, etc.) use your Astra Token, and actions specific to a Pulsar tenant (message namespaces, topics, message metrics, etc.) use a Pulsar JWT token.

If you would like to get a little deeper into exactly which token covers what action, see the following documentation:

Geo-replication is available to both Pay As You Go and dedicated serverless environments. Both can replicate to other clusters, but there are some differences.

In Pay As You Go streaming, traffic between clusters is secured over the (public) internet, while dedicated clusters have the option for private communication. Pay As You Go environments can replicate between any supported region of the same cloud provider. With dedicated clusters, you can use almost any region supported by your cloud provider, as well as across cloud providers. Email the team for more information.

For more on geo-replication, see Geo-replication.

Unless you are starting a project from scratch, you likely have message data that needs to be brought over to your Astra Streaming tenant(s). The Streaming Team has quite a bit of experience with this and can help you find the right way to migrate. Email the team for more information.

It is common to have a hierarchy of development environments which app changes are promoted through before reaching production. The configurations of middleware and platforms supporting the app should be kept in parity to promote stability and fast iterations with low volatility.

To support the hierarchy of development environments pattern, we recommend using Tenants to represent each development environment. This gives you the greatest flexibility to balance a separation of roles with consistent service configuration. All tokens created within a Tenant are limited to that Tenant.

For example, start with a tenant named “Dev” that development teams have access to (and create tokens from), then create other tenants named “Staging” and “Production”. Each Tenant has progressively less permissions to create tokens, but maintains parity between the three running environments.

Alternatively, you might choose to separate development environments by namespace within your Astra Streaming tenant. While this doesn’t offer as much flexibility as separation by tenant, it does offer a much simpler model to manage. Also, note that in this scheme you cannot limit access by namespace. All tokens would have access to all namespaces.

As mentioned previously, Astra Streaming is actively maintained to keep parity with the official Apache Pulsar project. The notable differences arise from accessibility and security. Naturally, you have less control in a managed, serverless cluster than you do in a cluster running in your own environment. Beyond those differences, the effort to develop locally and then move to Astra Streaming should not be significant, but it is recommended to develop directly in Astra Streaming. If you are trying to save costs, use the free tier of Astra Streaming and then “switch” when you are ready to stage your production services.

Yes, Astra Streaming offers a fully compatible Kafka and RabbitMQ implementation. This means you can use your existing Kafka or RabbitMQ applications with Astra Streaming. You can also use the Astra Streaming Kafka or RabbitMQ implementation with your existing Pulsar applications. Astra Streaming comes with custom ports for Kafka and RabbitMQ workloads. Learn more about the Starlight suite of APIs.

Yes, Astra Streaming offers a fully compatible JMS implementation. This means you can use your existing JMS applications with Astra Streaming. You can also use the Astra Streaming JMS implementation with your existing Pulsar applications. Learn more about the Starlight suite of APIs.