Astra Streaming FAQs
When considering Astra Streaming for your production workloads, you might have questions about what options are available for connecting and consuming your serverless clusters. This page answers some common questions about getting started with and using Astra Streaming.
Why is Astra Streaming based on Apache Pulsar™?
For information about the decision to use Apache Pulsar, see Four Reasons Why Apache Pulsar is Essential to the Modern Data Stack.
What happened to Kesque?
Astra Streaming is based heavily on technology originally created as part of Kesque. With the launch of Astra Streaming, DataStax began shutting down the Kesque service and migrated customers to Astra Streaming.
Who should use Astra Streaming?
Astra Streaming has been architected to satisfy the most stringent enterprise requirements around availability, scale, and latency. Astra Streaming was built to handle mission critical use cases for Fortune 100 companies across BFSI, Telecommunications, Technology, Retail, Oil and Gas, and Healthcare.
The pricing model also makes Astra Streaming accessible to mid market and small/medium business customers who need event stream processing capabilities to run core parts of their business.
And finally Astra Streaming offers a user-friendly interface and free tier to satisfy the needs of individual developers and technologists who want to learn more about Apache Pulsar and data streaming in general.
Why does DataStax call Astra Streaming "serverless"?
Running a Apache Pulsar™ cluster that can handle at-scale workloads is not a trivial task. It requires many (virtual) machines to be configured in a very particular way.
In traditional cloud environments, you would pay hourly for every machine whether they are being used for workloads or not, and you would carry the burden of maintaining the server infrastructure.
A serverless environment removes those operational burdens, and you pay only for the resources you actually use.
What is CDC for Astra DB?
Change Data Capture (CDC) for Astra DB uses a Pulsar IO source connector that processes changes from the Cassandra Change Agent through a Pulsar topic. For more information, see Enable Change Data Capture for Astra DB.
What are tenants, namespaces, topics, and sinks?
Tenants are an isolated administrative unit for which an authorization scheme can be set and a set of clusters can be defined. Each tenant can have multiple namespaces, a logical container for creating and managing a hierarchy of topics. A topic is a named channel for transmitting messages from producers to consumers. A sink feeds data from Astra Streaming to an external system, such as Apache Cassandra® or Elasticsearch.
Shared clusters and Streaming Capacity Units (dedicated clusters)
The following questions address differences between shared clusters and Streaming Capacity Units.
What happened to dedicated clusters?
Astra Streaming dedicated clusters are now called Streaming Capacity Units.
Are connections to shared clusters less secure than Streaming Capacity Units?
Connections to both cluster types are secure. However, Streaming Capacity Units offer the option for private connectivity.
For Streaming Capacity Units, all connections in Astra Streaming are guarded by authentication, authorization, and secure (TLS) communications. You can connect over the public internet or establish a private connection.
Shared clusters also use a secure connection over the public internet. However, shared clusters don’t support private links.
For more information, see Astra Streaming pricing.
What are the differences in geo-replication for shared clusters and Streaming Capacity Units?
Geo-replication is available for both shared clusters and Streaming Capacity Units. Both cluster types can replicate to other clusters, but there are some differences:
- Connections
-
For shared clusters, traffic between clusters is secured over the public internet.
Streaming Capacity Units can use either the default secure public internet connection or enable private links.
- Regions
-
Shared clusters can replicate between any supported region of the same cloud provider.
Streaming Capacity Units can use almost any region supported by your cloud provider, as well as across cloud providers.
For more on geo-replication, see Geo-replication.
Security and access control
The following questions address security and access control in Astra Streaming.
Does Astra Streaming support single-sign on?
If your Astra organization is on the Pay As You Go or Enterprise Astra subscription plan, your users can use SSO to sign in to the Astra Portal. For more information, see Configure single sign-on for Astra.
Application development
The following questions are related to application development with Astra Streaming.
Can I migrate data from my existing Pulsar cluster to Astra Streaming?
Unless you are starting a project from scratch, you likely have message data that needs to be brought over to your Astra Streaming tenants. For migration assistance, contact DataStax Support.
Every tenant in Astra Streaming comes with custom ports for Kafka and RabbitMQ workloads. DataStax also offers a fully-compatible JMS implementation for your Java workloads. For more information, see DataStax Starlight Suite of Apache Pulsar™ Extensions.
How do I separate messaging traffic?
It is common to have a hierarchy of development environments through which you promote app changes before they reach production. The configurations of middleware and platforms supporting the app should be kept in parity to promote stability and fast iterations with low volatility.
- By Tenant
-
To support the hierarchy of development environments, DataStax recommends creating separate tenants for each development environment. This gives you the greatest flexibility to balance the separation of roles with consistent service configuration.
All tokens created within a tenant are limited to that tenant.
For example, start with a tenant named
Devthat your development teams can access and create tokens for, and then create other tenants namedStagingandProduction. At each level of the hierarchy, there are fewer users with access to the environment’s tenant, which means fewer opportunities to create tokens that can programmatically access that tenant. Yet, you still maintain parity across the three environments. - By Namespace
-
Alternatively, you might choose to separate development environments by namespace within your Astra Streaming tenant. While this doesn’t offer as much flexibility as separation by tenant, it does offer a much simpler model to manage. Also, note that in this scheme you cannot limit access by namespace. All tokens would have access to all namespaces.
Can I develop applications on open source Pulsar and then move to Astra Streaming?
Astra Streaming is actively maintained to keep parity with the official Apache Pulsar project. The notable differences arise from accessibility and security. Naturally, you have less control in a managed, serverless cluster than you do in a cluster running in your own environment. Beyond those differences, the effort to develop locally and then move to Astra Streaming should not be significant, but DataStax recommends that you develop directly in Astra Streaming. If you are trying to reduce costs, use the free tier of Astra Streaming and then switch when you are ready to stage your production services.
Can I use Astra Streaming with my existing Kafka or RabbitMQ applications?
Yes, Astra Streaming offers a fully compatible Kafka and RabbitMQ implementation. This means you can use your existing Kafka or RabbitMQ applications with Astra Streaming. You can also use the Astra Streaming Kafka or RabbitMQ implementation with your existing Pulsar applications. Astra Streaming comes with custom ports for Kafka and RabbitMQ workloads. Learn more about the Starlight suite of APIs.
Can I use Astra Streaming with my existing Java applications?
Yes, Astra Streaming offers a fully compatible JMS implementation. This means you can use your existing JMS applications with Astra Streaming. You can also use the Astra Streaming JMS implementation with your existing Pulsar applications. Learn more about the Starlight suite of APIs.
