Connectors

Astra Streaming offers fully-managed Apache Pulsar™ connectors.

Create, monitor, and manage both source and sink connectors through our simple UI, the pulsar-admin CLI, or RESTful API. Connect popular data sources to Apache Pulsar topics or sink data from Pulsar topics to popular systems.

Below is a list of Apache Pulsar source and sink connectors supported by Astra Streaming.

Astra Streaming does not support custom sink or source connectors.

Sink Connectors

AstraDB sink

The AstraDB sink connector reads messages from Apache Pulsar topics and writes them to AstraDB systems.

AstraDB sink documentation.

Cloud Storage sink

The Cloud Storage sink connector reads messages from Apache Pulsar topics and writes them to Cloud Storage systems.

Cloud Storage sink documentation.

ElasticSearch sink

The Elasticsearch sink connector reads messages from Apache Pulsar topics and writes them to Elasticsearch systems.

Elasticsearch sink documentation.

Google BigQuery sink

The Google BigQuery sink connector reads messages from Apache Pulsar topics and writes them to BigQuery systems.

Google BigQuery sink documentation.

JDBC-Clickhouse sink

The JDBC-ClickHouse sink connector reads messages from Apache Pulsar topics and writes them to JDBC-ClickHouse systems.

JDBC ClickHouse sink documentation.

JDBC-MariaDB sink

The JDBC-MariaDB sink connector reads messages from Apache Pulsar topics and writes them to JDBC-MariaDB systems.

JDBC MariaDB sink documentation.

JDBC-PostgreSQL sink

The JDBC-PostgreSQL sink connector reads messages from Apache Pulsar topics and writes them to JDBC-PostgreSQL systems.

JDBC PostgreSQL sink documentation.

JDBC-SQLite

The JDBC-SQLite sink connector reads messages from Apache Pulsar topics and writes them to JDBC-SQLite systems.

JDBC SQLite sink documentation.

Kafka

The Kafka sink connector reads messages from Apache Pulsar topics and writes them to Kafka systems.

Kafka sink documentation.

Kinesis

The Kinesis sink connector reads messages from Apache Pulsar topics and writes them to Kinesis systems.

Kinesis sink documentation.

Snowflake

The Snowflake sink connector reads messages from Apache Pulsar topics and writes them to Snowflake systems.

Snowflake sink documentation.

Source Connectors

Data Generator source

The Data generator source connector produces messages for testing and persists the messages to Pulsar topics.

Data Generator source documentation

Debezium MongoDB source

The Debezium MongoDB source connector reads data from Debezium MongoDB systems and produces data to Pulsar topics.

Debezium MongoDB source documentation

Debezium MySQL source

The Debezium MySQL source connector reads data from Debezium MySQL systems and produces data to Pulsar topics.

Debezium MySQL source documentation

Debezium Oracle source

The Debezium Oracle source connector reads data from Debezium Oracle systems and produces data to Pulsar topics.

Debezium Oracle source documentation

Debezium Postgres source

The Debezium PostgreSQL source connector reads data from Debezium PostgreSQL systems and produces data to Pulsar topics.

Debezium PostgreSQL source documentation

Debezium SQL Server source

The Debezium SQL Server source connector reads data from Debezium SQL Server systems and produces data to Pulsar topics.

Debezium SQL Server source documentation

Kafka source

The Kafka source connector reads data from Kafka systems and produces data to Pulsar topics.

Kafka source connector documentation

AWS Kinesis source

The AWS Kinesis source connector reads data from Kinesis systems and produces data to Pulsar topics.

Kinesis source connector documentation

Experimental Connectors

DataStax is always experimenting with connectors. Below are the connectors currently in development that have not yet been promoted to official support in Astra Streaming.

To get access to these connectors, contact DataStax Support.

Sink Connectors (experimental)

Kinetica
Aerospike
Azure DocumentDB
Azure Data Explorer (Kusto)
Batch Data Generator
CoAP
Couchbase
DataDog
Diffusion
Flume
Apache Geode
Hazelcast
Apache HBase
HDFS 2
HDFS 3
Humio
InfluxDB
JMS
Apache Kudu
MarkLogic
MongoDB
MQTT
Neo4J
New Relic
OrientDB
Apache Phoenix
PLC4X
RabbitMQ
Redis
SAP HANA
SingleStore
Apache Solr
Splunk
XTDB
Zeebe

Source Connectors (experimental)

Cassandra Source
Kinetica
Azure DocumentDB
Batch Data Generator
Big Query
canal
CoAP
Couchbase
datadog
diffusion
DynamoDB
file
flume
Apache Geode
Hazelcast
Humio
JMS
Apache Kudu
MarkLogic
MongoDB
MQTT
Neo4J
New Relic
NSQ
OrientDB
Apache Phoenix
PLC4X
RabbitMQ
Redis
SAP HANA
SingleStore
Splunk
Twitter
XTDB
Zeebe

Listing Sink Connectors

To list available sink connectors in your Astra Streaming tenant, use any of the following.

Pulsar Admin
curl

Assuming you have downloaded client.conf to the Pulsar folder:

./bin/pulsar-admin sinks available-sinks

You need a Pulsar token for REST API authentication. This is different from your Astra DB application tokens.

In the Astra Portal header, click Applications, and then select Streaming.
Click your tenant’s name, and then click the Settings tab.
Click Create Token.
Copy the token, store it securely, and then click Close.
Click the Connect tab, and then copy the Web Service URL.

Create environment variables for your tenant’s token and web service URL:

export WEB_SERVICE_URL=<replace-me>
export ASTRA_STREAMING_TOKEN=<replace-me>

Use these values to form curl commands to the REST API:

curl "$WEB_SERVICE_URL/admin/v3/sinks/builtinsinks" -H "Authorization: $ASTRA_STREAMING_TOKEN"

Listing Source Connectors

To list available source connectors in your Astra Streaming tenant, use any of the following.

Pulsar Admin
cURL