Create project

pom file

Create a maven project or module and add a rocketmq client dependency.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>cn.tedu</groupId>
    <artifactId>rocketmq-api</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-client</artifactId>
            <version>4.7.1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-store</artifactId>
            <version>4.7.1</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Synchronization message

Strong consistency should be ensured when sending synchronization messages. Feedback information will not be sent to the producer until the messages sent to the master are copied to the slave.

This reliable synchronous sending method is widely used, such as important message notification and short message notification.

producer

package m1;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.util.Scanner;

public class Producer {
    public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException, MQBrokerException {
        //New producer instance
        DefaultMQProducer p = new DefaultMQProducer("producer1");
        //Set name server address
        p.setNamesrvAddr("192.168.64.141:9876");
        //Start producer connection server
        p.start();

        //Encapsulate Message data into Message objects
        //send out
        while(true){
            System.out.println("Enter message:");
            String s = new Scanner(System.in).nextLine();
            /**
             * Topic --- Primary classification
             * Tag --- Secondary classification (optional)
             */
            //Destination, to whom, message sent
            Message msg = new Message("Topic1", "TagA", s.getBytes());//Topic1 was created on the server itself
            SendResult r = p.send(msg);
            System.out.println("Messages sent:"+r);
        }
    }
}

consumer

Key points for consumers:

1.push and pull

Consumers have two modes: push and pull.

In push mode, the server actively sends messages to consumers; In pull mode, consumers actively request messages from the server.

When the processing capacity of consumers is limited, in order to reduce the pressure of consumers, the pull mode can be adopted. Pull mode is used in most cases.

2.NameServer

The consumer needs to ask the NameServer for the routing information of the Topic.

3.Topic

Receives a message from the specified topic. Topic is equivalent to primary classification.

4.Tag

Topic is equivalent to level 1 Classification, and Tag is equivalent to level 2 classification.

• Multiple tags can be written as taga | Tag B | TAGC
• If you do not specify tags or receive all tags, you can write asterisks:*

package m1;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.*;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

public class Consumer {
    public static void main(String[] args) throws MQClientException {
        //New consumer instance
        DefaultMQPushConsumer c = new DefaultMQPushConsumer("Consumer1");

        //Set name server
        c.setNamesrvAddr("192.168.64.141:9876");

        //Subscribe to messages (from which)
        /**
         * Label settings:
         *      TagA
         *      TagB || TagC || TagD
         *      *
         */
        c.subscribe("Topic1", "TagA");//Subscribe to the message of tag A from topic1

        //Message listener
        //The concurrent listener starts multiple threads and can process multiple messages in parallel
        c.setMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
                for (MessageExt ext : msgs) {//Shortcut key: msgs.for
                    String s = new String(ext.getBody());
                    System.out.println("received:"+s);
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                //return ConsumeConcurrentlyStatus.RECONSUME_LATER;
            }
        });
        
        //start-up
        c.start();
    }
}

Asynchronous message

After receiving the message, the master immediately gives feedback to the producer. The message is then copied asynchronously to the slave.

Asynchronous messages are usually used in business scenarios that are sensitive to response time, that is, the sender cannot tolerate waiting for a Broker's response for a long time.

producer

package demo2;

import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendCallback;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.util.Scanner;

/*
Send message asynchronously

After a message is sent, you don't have to wait for the server's feedback on the message, but you can send subsequent messages immediately
 Use a listener to receive feedback from the server in an asynchronous manner
 */
public class Producer {
    public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException {
        DefaultMQProducer p = new DefaultMQProducer("producer-demo2");
        p.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");
        p.start();

        p.setRetryTimesWhenSendAsyncFailed(0);

        String topic = "Topic2";
        String tag = "TagA";
        String key = "Key-demo2";


        while (true) {
            System.out.print("Input message,Separate multiple messages with commas: ");
            String[] a = new Scanner(System.in).nextLine().split(",");

            for (String s : a) {
                Message msg = new Message(topic, tag, key, s.getBytes());

                p.send(msg, new SendCallback() {
                    @Override
                    public void onSuccess(SendResult sendResult) {
                        System.out.println("\n\n Message sent successfully : "+sendResult);
                    }

                    @Override
                    public void onException(Throwable throwable) {
                        System.out.println("\n\n Message sending failed");
                    }
                });

                System.out.println("--------------------Message sent-----------------------");
            }
        }
    }
}

consumer

package demo2;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;
/*
Exactly the same as demo1.Consumer
 */
public class Consumer {
    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer c = new DefaultMQPushConsumer("consumer-demo2");
        c.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");

        c.subscribe("Topic2", "TagA");

        c.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                for (MessageExt msg : list) {
                    System.out.println(new String(msg.getBody()) + " - " + msg);
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

        c.start();
        System.out.println("Start consumption data");
    }
}

One way message

This method is mainly used in scenarios that do not particularly care about sending results, such as log sending.

producer

package demo3;

import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.util.Scanner;

/*
One way message

After the message is sent, the server will not return results
 */
public class Producer {
    public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException {
        DefaultMQProducer p = new DefaultMQProducer("producer-demo3");
        p.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");
        p.start();

        String topic = "Topic3";
        String tag = "TagA";

        while (true) {
            System.out.print("Input message,Separate multiple messages with commas: ");
            String[] a = new Scanner(System.in).nextLine().split(",");
            for (String s : a) {
                Message msg = new Message(topic, tag, s.getBytes());
                p.sendOneway(msg);
            }
            System.out.println("--------------------Message sent-----------------------");
        }
    }
}

consumer

package demo3;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

/*
Exactly the same as demo1.Consumer
 */
public class Consumer {
    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer c = new DefaultMQPushConsumer("consumer-demo2");
        c.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");

        c.subscribe("Topic3", "TagA");

        c.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                for (MessageExt msg : list) {
                    System.out.println(new String(msg.getBody()) + " - " + msg);
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

        c.start();
        System.out.println("Start consumption data");
    }
}

Sequential message

The figure above illustrates the basic principle of Rocketmq sequential message:

• The same set of ordered message sequences will be sent to the same queue and processed in the way of FIFO
• A queue allows only one consumer thread to receive messages, which ensures that messages are received in order

Take the order as an example:

The sequential process of an order is: create, pay, push and complete. Messages with the same order number will be sent to the same queue successively. When consuming, the message of the same order is received from the same queue.

producer

package demo4;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.MessageQueueSelector;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageQueue;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.util.List;
import java.util.Scanner;
/*
For the following messages, messages with the same id are sent to the same queue in sequence,
Consumption is also consumed sequentially from the same queue
                                              topic

                                        =======================  queue1
                                        =======================  queue2
111,Message 1 111, message 2 111, message 3 ------ > ========================================== queue3
                                        =======================  queue4
222,Message 1 222, message 2 222, message 3 ------ > ======================================== queue5
                                        =======================  queue6
333,Message 1 333, message 2 333, message 3 ------ > ======================================== queue7
                                        =======================  queue8
                                                    ......
 */
public class Producer {
    static String[] msgs = {
            "15103111039,establish",
                                "15103111065,establish",
            "15103111039,payment",
                                                    "15103117235,establish",
                                "15103111065,payment",
                                                    "15103117235,payment",
                                "15103111065,complete",
            "15103111039,Push",
                                                    "15103117235,complete",
            "15103111039,complete"
    };

    public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException, MQBrokerException {
        DefaultMQProducer p = new DefaultMQProducer("producer-demo4");
        p.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");
        p.start();

        String topic = "Topic4";
        String tag = "TagA";

        for (String s : msgs) {
            System.out.println("Press enter to send this message: "+s);
            new Scanner(System.in).nextLine();

            Message msg = new Message(topic, tag, s.getBytes());

            String[] a = s.split(",");
            long orderId = Long.parseLong(a[0]);

            /*
            MessageQueueSelector Used to select the sending queue,
            Here, the queue index is calculated by taking the remainder of the queue number with the order id

            send(msg, queueSelector, obj)
            The third parameter is passed to the queueSelector as its third parameter
             */
            SendResult r = p.send(msg, new MessageQueueSelector() {
                /*
                Meaning of three parameters:
                queueList: List of all queues in the current Topic
                message: news
                o: send()orderId passed in by method
                 */
                @Override
                public MessageQueue select(List<MessageQueue> queueList, Message message, Object o) {
                    Long orderId = (Long) o;
                    //The order id is the remainder of the queue quantity, and the same order id gets the same queue index
                    long index = orderId % queueList.size();
                    System.out.println("Message sent to: "+queueList.get((int) index));
                    return queueList.get((int) index);
                }
            }, orderId);

            System.out.println(r+"\n\n");
        }
    }
}

consumer

package demo4;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

public class Consumer {
    public static void main(String[] args) throws MQClientException {
        DefaultMQPushConsumer c = new DefaultMQPushConsumer("consumer-demo4");
        c.setNamesrvAddr("192.168.64.151:9876;192.168.64.152:9876");
        
        c.subscribe("Topic4", "*");

        c.registerMessageListener(new MessageListenerOrderly() {
            @Override
            public ConsumeOrderlyStatus consumeMessage(List<MessageExt> list, ConsumeOrderlyContext consumeOrderlyContext) {
                String t = Thread.currentThread().getName();

                for (MessageExt msg : list) {
                    System.out.println(t+" - "+ msg.getQueueId() + " - " +new String(msg.getBody()));
                }

                return ConsumeOrderlyStatus.SUCCESS;
            }
        });

        c.start();
        System.out.println("Start consumption data");
    }
}

Delay message

After the message is sent to the Rocketmq server, it will be delivered to the consumer after a certain delay.

Usage scenario of delay message:

For example, in e-commerce, you can send a delay message after submitting an order, check the status of the order after 1h, and cancel the order and release the inventory if it is still unpaid.

When the producer sends a message, set the message delay:

msg.setDelayTimeLevel(3);

Where 3 represents level rather than a specific time value. The corresponding relationship between level and delay duration is defined in MessageStoreConfig class:

this.messageDelayLevel = "1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h";

Correspondence table:

producer

package m1;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;

import java.util.Scanner;

public class Producer {
    public static void main(String[] args) throws MQClientException, RemotingException, InterruptedException, MQBrokerException {
        //New producer instance
        DefaultMQProducer p = new DefaultMQProducer("producer1");
        //Set name server address
        p.setNamesrvAddr("192.168.64.141:9876");
        //Start producer connection server
        p.start();

        //Encapsulate Message data into Message objects
        //send out
        while(true){
            System.out.println("Enter message:");
            String s = new Scanner(System.in).nextLine();
            /**
             * Topic --- Primary classification
             * Tag --- Secondary classification (optional)
             */
            //Destination, to whom, message sent
            Message msg = new Message("Topic1", "TagA", s.getBytes());//Topic1 was created on the server itself
            if(Math.random() < 0.5){
                msg.setDelayTimeLevel(3);//Represents 10 seconds
                System.out.println("This message is delayed by 10 seconds");
            }
            SendResult r = p.send(msg);
            System.out.println("Messages sent:"+r);
        }
    }
}

consumer

package m1;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.*;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

/**
 * @Authod yuhe
 * Create 2021-10-29-16:40
 */
public class Consumer {
    public static void main(String[] args) throws MQClientException {
        //New consumer instance
        DefaultMQPushConsumer c = new DefaultMQPushConsumer("Consumer1");

        //Set name server
        c.setNamesrvAddr("192.168.64.141:9876");

        //Subscribe to messages (from which)
        /**
         * Label settings:
         *      TagA
         *      TagB || TagC || TagD
         *      *
         */
        c.subscribe("Topic1", "TagA");//Subscribe to the message of tag A from topic1

        //Message listener
        //The concurrent listener starts multiple threads and can process multiple messages in parallel
        c.setMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
                for (MessageExt ext : msgs) {//Shortcut key: msgs.for
                    String s = new String(ext.getBody());
                    System.out.println("received:"+s);
                }
                //return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                /**
                 * When the message processing fails, you can tell the server to resend the message and consume again later.
                 * If multiple processing fails, the maximum number of retries will be 18 (18 delay levels), and the retry interval will be longer and longer
                 */
                return ConsumeConcurrentlyStatus.RECONSUME_LATER;
            }
        });

        //start-up
        c.start();
    }
}

Batch message

Sending messages in bulk can significantly improve the performance of delivering small messages. The limitation is that these batch messages should have the same topic, the same waitStoreMsgOK, and cannot be delayed messages. In addition, the total size of this batch of messages should not exceed 4MB.

producer

consumer

Message filtering

Tag filtering

Tag can meet the needs of most message filtering. Using tag filtering is very simple, for example:

consumer.subscribe("Topic1", "TagA || TagB || TagC");

Filter custom attributes

Producers can add custom attributes to messages:

msg.putUserProperty("prop1", "1");
msg.putUserProperty("prop2", "2");

When consumers receive data, they can filter messages according to attributes:

consumer.subscribe("Topic7", MessageSelector.bySql("prop1=1 or prop2=2"));

You can see that the filtering syntax of custom attributes is Sql syntax. RocketMQ only defines some basic syntax to support this feature. The supported Sql filtering syntax is as follows:

• Numerical comparison, such as: >, > =, <, < =, BETWEEN, =;
• Character comparison, such as: =, < >, IN;
• IS NULL or IS NOT NULL;
• Logical symbols AND, OR, NOT;

producer

consumer

Transaction message

RocketMQ provides reliability messages, also known as transaction messages. Let's analyze its principle.

Principle of transaction message

Let's look at how RocketMQ's transaction messages send "reliable messages". We only need the following three steps:

1. Send half message (half message will not be sent to consumers)
2. Execute local transactions
3. Submit message

After the transaction message is sent, consumers can consume data in a normal way.

RocketMQ's automatic retransmission mechanism can ensure that messages are consumed correctly in most cases.

If the final consumption of the message fails, it can also be handled manually.
The above analysis is the execution process under normal conditions. Here are two error cases:

1. Rollback message when transaction execution fails
2. When the server cannot know the message status, it needs to actively check the message status

Rollback:

Message check back:

producer

consumer