Consuming change data with Apache Pulsar

This article is a continuation of the CDC with Cassandra and Pulsar article. Please read that article first to understand the fundamentals of what resources are being used.

Pulsar clients

Each client handles message consumption a little differently but there is one overall pattern to follow. As we learned in the previous sections, a CDC message will arrive as an Avro GenericRecord of type KeyValue. Typically, the first step will be to separate the key and value portions of the message. You will find the Cassandra table’s key fields in the key of the record and the change data in the value portion of the record. Both of which are Avro records themselves. From there you’ll want to deserialize the Avro record and extract the interesting info.

Below are example implementations for each runtime consuming messages from the CDC data topic.

While these examples are in the “astra-streaming-examples” repository, they are not Astra specific. You can use these examples to consume CDC data topics in your own Cassandra/Pulsar clusters.

csharp icon

C#

golang icon

Golang

java icon

Java

node icon

Node.js

python icon

Python

Pulsar functions

It is very common to have a function consuming the CDC data. Functions usually perform additional processing on the data and pass it to another topic. Similar to a client consumer, it will need to deserialize the message data. Below are examples of different functions consuming messages from the CDC data topic.

While these examples are in the “astra-streaming-examples” repository, they are not Astra specific. You can use these examples to consume CDC data topics in your own Cassandra/Pulsar clusters.

golang icon

Golang

java icon

Java

python icon

Python

Next

You’re ready to tackle CDC like a pro! Use our CDC questions and patterns as reference as you near production.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com