Enabling server-side filters
Starlight for JMS version 3.x offers server-side filtering, which executes JMS filtering features on the Pulsar Broker instead of the Client.
Before 3.x, filtering was implemented with
jms.enableClientSideEmulation=true on the Client side, which meant the Client application received all messages, even if they didn’t match the filter. This caused inefficient network, CPU, and memory usage.
Server-side filtering offloads the filtering to the Broker nodes and includes the following JMS filtering features:
Broker-side message selectors - The Broker filters messages instead of the Client
timeToLive- each message can have a time to live value
noLocalfilter - Filter out messages sent from the same connection
Server-side filtering requires Pulsar 2.10.1+ or Luna Streaming 2.10+.
.nar file filter is packaged with Luna Streaming 2.10.03+ and also available in the GitHub repo here.
Install Starlight for JMS version 3.x with the fat JAR that includes all dependencies. Put the
The path for the .nar is relative to your Pulsar directory.
For more, see Standalone quick start.
PulsarConnectionFactoryin Starlight for JMS to disable client-side emulation and use server-side filtering:
jms.enableClientSideEmulation = false; jms.useServerSideFiltering = true;
entryFilterNamesand to look for the
You’re ready to use server-side filtering.
Starlight for JMS allows you to pre-configure the
selector property when creating your Pulsar subscription. Modify
PulsarConnectionFactory in Starlight for JMS to include:
jms.filtering=true jms.selector=YOUR MESSAGE SELECTOR
If you have deployed the
.nar file and configured the
entryFiltersDirectory properties in
broker.conf, the Broker will begin filtering messages automatically.
This only partially works for a non-JMS client, because batch messages cannot be completely filtered out by the Broker. If you use this feature, you should use the Starlight for JMS 2.x client and the
In this case, the server-side filter applies both to JMS queues and JMS topics. The JMS client automatically downloads the
jms.selector from the subscription and applies it locally, where batch message filtering may have partially accepted the message on the Broker.
If you set
Batch messages - Server-side filtering works on the Entry level in Pulsar. An Entry is a single message or a single batch of messages. The Broker can’t filter out a few messages in a batch; it either passes completely or fails completely.
For example, if a batch message is partially accepted by the Broker filter, the Broker will still send the entire batch to the Client, and the Client will execute the filter locally. If you don’t want the Client to receive unwanted messages, disable batching on your producers.
Pulsar IO Sources enable batching and compression by default.
For more, see Batch index acknowledgement.
Increased Broker resources usage - The Pulsar Broker does not perform processing on messages by default. When server-side filtering is enabled, the Broker will require additional CPU and heap memory to process filters.
DataStax recommends disabling compression and batching for messages that will be filtered on the Broker. Batch messages with compression will consume a non-negligible amount of CPU, heap, and direct memory, because filtering requires decompressing the batch to access Message Properties. This does not apply to compressed single messages.
Unsupported message types - Starlight for JMS does not support encrypted batch messages because the filter can’t decode the payload of the entry. For the same reason, Starlight for JMS does not support custom formats like native Kafka messages.
Starlight for JMS standalone quick start: Create a simple command line Java JMS client that connects to a local Pulsar installation.
Installing Starlight for JMS: Install Starlight for JMS in your own JMS project.
Mapping Pulsar concepts to JMS specifications: Understand Pulsar concepts in the context of JMS.
Starlight for JMS implementation details: Understand key implementation details for Starlight for JMS.
Starlight for JMS FAQs: Frequently asked questions about Starlight for JMS.
Starlight for JMS configuration reference: Starlight for JMS configuration reference.