Article catalog
Message queue RocketMQ: (I) overview
Message queuing RocketMQ: (II) system architecture
Message queue RocketMQ: (III) sending ordinary messages (three methods)
Message queue RocketMQ: (IV) sequential messages
Message queue RocketMQ: (V) delay message
Message queue RocketMQ: (VII) batch messages
premise
Create a Maven Java project.
The Client dependency of RocketMQ is introduced, and the version needs to be consistent with the server.
<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-client</artifactId> <version>4.9.0</version> </dependency>
6, Message filtering
When subscribing to a message, a message maker can not only specify the Topic to subscribe to the message, but also filter the messages in the specified Topic according to the specified conditions, that is, he can subscribe to a message type with finer granularity than the Topic.
There are two filtering methods for the specified Topic message: Tag filtering and SQL filtering.
1. Tag filtering
Tag message tag, used to classify messages under Topic. When sending a message, the producer specifies the tag of the message, and the consumer subscribes according to the specified tag.
collocation method
- When sending a message, the message must indicate Tag
Message msg = new Message("LearnTopic","TagA","Hello RocketMQ".getBytes());
- Subscribe to all types of messages under Topic. Tag is marked with an asterisk*
consumer.subscribe("LearnTopic", "*");
- Subscribe to the next type of message under Topic, indicating the specific Tag
consumer.subscribe("LearnTopic", "TagA");
- Subscribe to various types of messages under Topic. Multiple tags are separated by 𞓜
consumer.subscribe("LearnTopic", "TagA || TagB");
Scene analysis
A series of messages will be generated during the process of e-commerce transaction from the customer placing an order to receiving the goods. The following messages are examples:
- Order message
- Payment message
- Logistics News
These messages are sent to Trade_Topic is subscribed to by different systems. Take the following systems as an example:
- Payment system: only subscribe to payment messages
- Logistics system: only subscribe to logistics messages
- Transaction success rate analysis system: subscription order and payment message
- Real time computing system: subscribe to all transaction messages
2. SQL filtering
SQL filtering is a way to filter user attributes embedded in messages through specific expressions. Through SQL filtering, you can realize complex filtering of messages. However, only consumers using PUSH mode can use SQL filtering.
SQL filter expression
Multiple constant types and operators are supported in SQL filter expressions.
Supported constant types:
- Value: for example: 123, 3.1415
- Characters: must be enclosed in single quotes, such as' abc '
- Boolean: TRUE or FALSE
- NULL: special constant, indicating NULL
Supported operators are:
- Value comparison: >, > =, <, < =, BETWEEN=
- Character comparison: =, < >, IN
- Logical operations: AND, OR, NOT
- NULL judgment: IS NULL or IS NOT NULL
be careful
By default, the Broker does not enable the SQL filtering function of messages. The following attributes need to be added to the configuration file loaded by the Broker to enable this function:
enablePropertyFilter = true
This modified configuration file needs to be specified when starting the Broker.
For example, for the startup of a stand-alone Broker, the modified configuration file is conf / Broker Conf, use the following command when starting:
sh bin/mqbroker -n localhost:9876 -c conf/broker.conf &
Scene analysis
E-commerce transactions produce a series of messages from customers placing orders to receiving goods. The messages are divided into order messages and logistics messages according to types. The logistics messages are defined with regional attributes, which are divided into Hangzhou and Shanghai according to regions:
- Order message
- Logistics News
- Logistics information and the region is Hangzhou
- Logistics information and the region is Shanghai
These messages are sent to Trade_Topic is subscribed to by different systems. Take the following systems as an example:
- Logistics system 1: only subscribe to logistics messages and the message region is Hangzhou
- Logistics system 2: only subscribe to logistics messages and the message region is Hangzhou or Shanghai
- Order tracking system: only subscribe to order messages
- Real time computing system: subscribe to all transaction messages
3. Code example (Tag filtering)
Tag filter Producer:
package com.learn.rocketmq.filter;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
public class FilterByTagProducer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("pg");
producer.setNamesrvAddr("localhost:9876");
producer.start();
String[] tags = {"TagA", "TagB", "TagC"};
for (int i = 0; i < 10; i++) {
byte[] body = ("Hello RocketMQ " + i).getBytes();
String tag = tags[i % tags.length];
Message msg = new Message("LearnTopic", tag, body);
SendResult sendResult = producer.send(msg);
System.out.println(sendResult);
}
producer.shutdown();
}
}
Tag filter Consumer:
package com.learn.rocketmq.filter;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;
public class FilterByTagConsumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("pg");
consumer.setNamesrvAddr("localhost:9876");
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
consumer.subscribe("LearnTopic", "TagA || TagB");
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
ConsumeConcurrentlyContext context) {
for (MessageExt me : msgs) {
System.out.println(me);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
System.out.println("Consumer Started");
}
}
4. Code example (SQL filtering)
SQL filter Producer:
package com.learn.rocketmq.filter;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
public class FilterBySQLProducer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("pg");
producer.setNamesrvAddr("localhost:9876");
producer.start();
for (int i = 0; i < 10; i++) {
try {
byte[] body = ("Hello RocketMQ " + i).getBytes();
Message msg = new Message("LearnTopic", "myTag", body);
// Embedding user attributes in advance
msg.putUserProperty("age", i + "");
SendResult sendResult = producer.send(msg);
System.out.println(sendResult);
} catch (Exception e) {
e.printStackTrace();
}
}
producer.shutdown();
}
}
SQL filter Consumer:
package com.learn.rocketmq.filter;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.MessageSelector;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;
public class FilterBySQLConsumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("pg");
consumer.setNamesrvAddr("localhost:9876");
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
// To filter out messages with age between [0, 6] from topic messages
consumer.subscribe("LearnTopic", MessageSelector.bySql("age between 0 and 6"));
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
ConsumeConcurrentlyContext context) {
for (MessageExt me : msgs) {
System.out.println(me);
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
System.out.println("Consumer Started");
}
}