Enabling server-side filters

Starlight for JMS offers server-side filtering, which executes JMS filtering features on the Pulsar Broker instead of the Client.
Previously, filtering was implemented with jms.enableClientSideEmulation=true on the Client side, which meant the Client application received all messages, even if they didn’t match the filter. This caused inefficient network, CPU, and memory usage.

Server-side filtering offloads the filtering to the Broker nodes and includes the following JMS filtering features:

Broker-side message selectors - The Broker filters messages instead of the Client
Per-message timeToLive - each message can have a time to live value
noLocal filter - Filter out messages sent from the same connection

Server-side filtering requires Pulsar 2.10.1+ or Luna Streaming 2.10+.

Enable server-side filtering

The .nar file filter is packaged with Luna Streaming 2.10.03+ and also available in the GitHub repo here.

Install Starlight for JMS with the fat JAR that includes all dependencies. Put the .nar file in /pulsar/filters.

The path for the .nar is relative to your Pulsar directory.

For more, see Standalone quick start.
Modify PulsarConnectionFactory in Starlight for JMS to disable client-side emulation and use server-side filtering:
```
jms.enableClientSideEmulation = false;
jms.useServerSideFiltering = true;
```
Modify broker.conf to accept entryFilterNames and to look for the .nar file in /pulsar/filters:

entryFilterNames=jms
entryFiltersDirectory=/pulsar/filters

You’re ready to use server-side filtering.

Configure per-subscription message selectors on the Broker

Starlight for JMS allows you to pre-configure the selector property when creating your Pulsar subscription. Modify PulsarConnectionFactory in Starlight for JMS to include:

jms.filtering=true
jms.selector=YOUR MESSAGE SELECTOR

If you have deployed the .nar file and configured the entryFilterNames and entryFiltersDirectory properties in broker.conf, the Broker will begin filtering messages automatically.

This only partially works for a non-JMS client, because batch messages cannot be completely filtered out by the Broker. If you use this feature, you should use the Starlight for JMS 2.x client and the jms.useServerSideFiltering feature.

In this case, the server-side filter applies both to JMS queues and JMS topics. The JMS client automatically downloads the jms.selector from the subscription and applies it locally, where batch message filtering may have partially accepted the message on the Broker.

If you set jms.selector on the subscription and set a message selector on the JMS client, the conditions on the subscription and the JMS client will both be fulfilled.

Limitations

Batch messages - Server-side filtering works on the Entry level in Pulsar. An Entry is a single message or a single batch of messages. The Broker can’t filter out a few messages in a batch; it either passes completely or fails completely.

For example, if a batch message is partially accepted by the Broker filter, the Broker will still send the entire batch to the Client, and the Client will execute the filter locally. If you don’t want the Client to receive unwanted messages, disable batching on your producers.

Pulsar IO Sources enable batching and compression by default.

For more, see Batch index acknowledgement.

Increased Broker resources usage - The Pulsar Broker does not perform processing on messages by default. When server-side filtering is enabled, the Broker will require additional CPU and heap memory to process filters.

DataStax recommends disabling compression and batching for messages that will be filtered on the Broker. Batch messages with compression will consume a non-negligible amount of CPU, heap, and direct memory, because filtering requires decompressing the batch to access Message Properties. This does not apply to compressed single messages.
Unsupported message types - Starlight for JMS does not support encrypted batch messages because the filter can’t decode the payload of the entry. For the same reason, Starlight for JMS does not support custom formats like native Kafka messages.

What’s next?

Starlight for JMS standalone quickstart: Create a simple command line Java JMS client that connects to a local Pulsar installation.
Installing Starlight for JMS: Install Starlight for JMS in your own JMS project.
Mapping Pulsar concepts to JMS specifications: Understand Pulsar concepts in the context of JMS.
Starlight for JMS implementation details: Understand key implementation details for Starlight for JMS.
Starlight for JMS FAQs: Frequently asked questions about Starlight for JMS.
Starlight for JMS configuration reference: Starlight for JMS configuration reference.
Starlight for JMS Github repo

Was this helpful?

Edit this page