Connectors
Astra Streaming offers fully-managed Apache Pulsar™ connectors.
Create, monitor, and manage both source and sink connectors through our simple UI, the pulsar-admin CLI, or RESTful API. Connect popular data sources to Apache Pulsar topics or sink data from Pulsar topics to popular systems.
Below is a list of Apache Pulsar source and sink connectors supported by Astra Streaming.
|
Astra Streaming does not support custom sink or source connectors. |
Sink Connectors
AstraDB sink
The AstraDB sink connector reads messages from Apache Pulsar topics and writes them to AstraDB systems.
Cloud Storage sink
The Cloud Storage sink connector reads messages from Apache Pulsar topics and writes them to Cloud Storage systems.
ElasticSearch sink
The Elasticsearch sink connector reads messages from Apache Pulsar topics and writes them to Elasticsearch systems.
Google BigQuery sink
The Google BigQuery sink connector reads messages from Apache Pulsar topics and writes them to BigQuery systems.
JDBC-Clickhouse sink
The JDBC-ClickHouse sink connector reads messages from Apache Pulsar topics and writes them to JDBC-ClickHouse systems.
JDBC-MariaDB sink
The JDBC-MariaDB sink connector reads messages from Apache Pulsar topics and writes them to JDBC-MariaDB systems.
JDBC-PostgreSQL sink
The JDBC-PostgreSQL sink connector reads messages from Apache Pulsar topics and writes them to JDBC-PostgreSQL systems.
JDBC-SQLite
The JDBC-SQLite sink connector reads messages from Apache Pulsar topics and writes them to JDBC-SQLite systems.
Kafka
The Kafka sink connector reads messages from Apache Pulsar topics and writes them to Kafka systems.
Kinesis
The Kinesis sink connector reads messages from Apache Pulsar topics and writes them to Kinesis systems.
Snowflake
The Snowflake sink connector reads messages from Apache Pulsar topics and writes them to Snowflake systems.
Source Connectors
Data Generator source
The Data generator source connector produces messages for testing and persists the messages to Pulsar topics.
Debezium MongoDB source
The Debezium MongoDB source connector reads data from Debezium MongoDB systems and produces data to Pulsar topics.
Debezium MySQL source
The Debezium MySQL source connector reads data from Debezium MySQL systems and produces data to Pulsar topics.
Debezium Oracle source
The Debezium Oracle source connector reads data from Debezium Oracle systems and produces data to Pulsar topics.
Debezium Postgres source
The Debezium PostgreSQL source connector reads data from Debezium PostgreSQL systems and produces data to Pulsar topics.
Debezium SQL Server source
The Debezium SQL Server source connector reads data from Debezium SQL Server systems and produces data to Pulsar topics.
Kafka source
The Kafka source connector reads data from Kafka systems and produces data to Pulsar topics.
AWS Kinesis source
The AWS Kinesis source connector reads data from Kinesis systems and produces data to Pulsar topics.
Experimental Connectors
DataStax is always experimenting with connectors. Below are the connectors currently in development that have not yet been promoted to official support in Astra Streaming.
If you would like to be given access to these connectors, please send a request to astrastreaming@datastax.com (include your Astra account UUID).
Sink Connectors (experimental)
Kinetica
Aerospike
Azure DocumentDB
Azure Data Explorer (Kusto)
Batch Data Generator
CoAP
Couchbase
DataDog
Diffusion
Flume
Apache Geode
Hazelcast
Apache HBase
HDFS 2
HDFS 3
Humio
InfluxDB
JMS
Apache Kudu
MarkLogic
MongoDB
MQTT
Neo4J
New Relic
OrientDB
Apache Phoenix
PLC4X
RabbitMQ
Redis
SAP HANA
SingleStore
Apache Solr
Splunk
XTDB
Zeebe
Source Connectors (experimental)
Cassandra Source
Kinetica
Azure DocumentDB
Batch Data Generator
Big Query
canal
CoAP
Couchbase
datadog
diffusion
DynamoDB
file
flume
Apache Geode
Hazelcast
Humio
JMS
Apache Kudu
MarkLogic
MongoDB
MQTT
Neo4J
New Relic
NSQ
OrientDB
Apache Phoenix
PLC4X
RabbitMQ
Redis
SAP HANA
SingleStore
Splunk
Twitter
XTDB
Zeebe
Listing Sink Connectors
To list available sink connectors in your Astra Streaming tenant, use any of the following.
-
Pulsar Admin
-
cURL
Assuming you have downloaded client.conf to the Pulsar folder:
./bin/pulsar-admin sinks available-sinksYou’ll need to create an Astra Streaming API token to be used with the Rest API. This is different from your Astra tokens.
Navigate to the "Settings" area in the Astra Streaming UI and choose "Create Token".
Retrieve the web service URL from the "Connect" tab in the Astra Streaming UI.
export WEB_SERVICE_URL=<replace-me>
export ASTRA_STREAMING_TOKEN=<replace-me>curl "$WEB_SERVICE_URL/admin/v3/sinks/builtinsinks" -H "Authorization: $ASTRA_STREAMING_TOKEN"Listing Source Connectors
To list available source connectors in your Astra Streaming tenant, use any of the following.
-
Pulsar Admin
-
cURL
Assuming you have downloaded client.conf to the Pulsar folder:
./bin/pulsar-admin sources available-sourcesYou’ll need to create an Astra Streaming API token to be used with the Rest API. This is different from your Astra tokens.
Navigate to the "Settings" area in the Astra Streaming UI and choose "Create Token".
Retrieve the web service URL from the "Connect" tab in the Astra Streaming UI.
export WEB_SERVICE_URL=<replace-me>
export ASTRA_STREAMING_TOKEN=<replace-me>curl "$WEB_SERVICE_URL/admin/v3/sources/builtinsources" -H "Authorization: $ASTRA_STREAMING_TOKEN"What’s next?
For more on Pulsar IO connectors, see the Pulsar documentation .
