Implementation of spring cloud Alibaba rocketmq asynchronous communication

Posted by Cleibe on Mon, 09 Dec 2019 05:36:31 +0100

This article discusses how to use RocketMQ Binder to subscribe and publish Spring Cloud application messages.

introduce

RocketMQ It is an open source distributed message system. Based on the highly available distributed cluster technology, it provides low latency and highly reliable message publishing and subscription services, which are widely used in many fields, including asynchronous communication decoupling, enterprise solutions, financial payment, telecommunications, e-commerce, express logistics, advertising and marketing, social networking, instant communication, mobile applications, mobile games, video Internet of things, Internet of vehicles, etc.

RocketMQ is an open-source distributed message middleware of Alibaba in 2012, which has been donated to the Apache Software Foundation and became the top project of Apache on September 25, 2017. As a domestic middleware that has experienced the baptism of "super project" of Alibaba double 11 for many times and has stable and outstanding performance, it has been used by more and more domestic enterprises in recent years with its high performance, low delay and high reliability.

RocketMQ features

It is a message middleware of queue model, with the characteristics of high performance, high reliability, high real-time, distributed, etc
Producer, Consumer and queue can be distributed
Producer sends messages to some queues in turn. The queue set is called Topic. If a Consumer does broadcast consumption, a Consumer instance consumes all queues corresponding to the Topic. If cluster consumption is done, multiple Consumer instances consume the queue set corresponding to the Topic on average
Can guarantee strict message order
Support pull and push message modes
Efficient subscriber level scalability
Real time message subscription mechanism
100 million message stacking capacity
Support multiple message protocols, such as JMS, OpenMessaging, etc
Less dependence

Spring Cloud Stream

Spring Cloud Stream is a framework for building message driven microservices.

Spring Cloud Stream provides a unified abstraction of the configuration of message middleware, and pubs / sub, consumer groups, semantics, and stateful partition are supported by these unified models.

The core components of Spring Cloud Stream include binders, binding s and messages. Applications interact with binders through inputs or outputs, and bind through our configuration. Binders interact with middleware. Message is the unified data specification format for data exchange.

Binding: includes Input Binding and Output Binding.

Binding provides a bridge between the message middleware and the Provider and Consumer provided by the application. It realizes that developers only need to use the Provider or Consumer of the application to produce or consume data, and blocks the contact between developers and the underlying message middleware.

Binder: a component integrated with external message middleware to create Binding. Each message middleware has its own binder implementation.

For example, Kafka implements KafkaMessageChannelBinder, RabbitMQ implements RabbitMessageChannelBinder, and RocketMQ implements rocketmgmessagechannelbinder.

Message: it is a module in the Spring Framework. Its function is the programming model of unified message.

For example, the corresponding model of message Messaging includes a message body Payload and message Header.

Spring cloud stream official website

Setting up and deploying RocketMQ in Window

download

The current latest version is 4.6.0

Download it and unzip it to: D:\rocketmq directory. It's better not to take space and too deep in the directory, otherwise the service may report an error

Start NameServer service

Before startup, you need to configure the system environment, otherwise an error will be reported.

Please set the ROCKETMQ_HOME variable in your environment!

System environment variable name: rocketmq · home

Configure environment variables according to the directory you unzipped. For example, my variable value is: D:\rocketmq

Enter the window command window, enter the directory D:\rocketmq\bin, and execute

start mqnamesrv.cmd

As above, NameServer starts successfully. Do not close the window during use.

Start Broker service

Enter bin directory and enter

start mqbroker.cmd -n localhost:9876

The above ip+port is the service address and port of rocketmq.

If you run the above command, the following error may be reported. Cannot find or load main class

If this happens, open bin -- > runbroker.cmd, modify% CLASSPATH% to% CLASSPATH%

Save to execute the above command again. After successful execution, prompt boot success to indicate success.

Example

This example implements the publishing and subscription receiving of three kinds of messages.

Create RocketMQ message producer

Create Ali rocketmq producer project with port 28081

pom.xml add dependency

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">

    <parent>
        <artifactId>cloud-alibaba</artifactId>
        <groupId>com.easy</groupId>
        <version>1.0.0</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <artifactId>ali-rocketmq-producer</artifactId>
    <packaging>jar</packaging>

    <dependencies>

        <!--rocketmq rely on-->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rocketmq</artifactId>
        </dependency>

        <!--web rely on-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Configure the Binding information of Output and validate it with @ EnableBinding annotation

application.yml configuration

server:
  port: 28081

spring:
  application:
    name: ali-rocketmq-producer
  cloud:
    stream:
      rocketmq:
        binder:
          # RocketMQ server address
          name-server: 127.0.0.1:9876
      bindings:
        output1: {destination: test-topic1, content-type: application/json}
        output2: {destination: test-topic2, content-type: application/json}

management:
  endpoints:
    web:
      exposure:
        include: '*'
  endpoint:
    health:
      show-details: always

ArProduceApplication.java

@SpringBootApplication
@EnableBinding({MySource.class})
public class ArProduceApplication {

    public static void main(String[] args) {
        SpringApplication.run(ArProduceApplication.class, args);
    }
}

Message producer service

MySource.java

package com.easy.arProduce;

import org.springframework.cloud.stream.annotation.Output;
import org.springframework.messaging.MessageChannel;

public interface MySource {

    @Output("output1")
    MessageChannel output1();

    @Output("output2")
    MessageChannel output2();
}

SenderService.java

package com.easy.arProduce;

import org.apache.rocketmq.spring.support.RocketMQHeaders;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageHeaders;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;
import org.springframework.util.MimeTypeUtils;

@Service
public class SenderService {

    @Autowired
    private MySource source;

    /**
     * Send string
     *
     * @param msg
     */
    public void send(String msg) {
        Message message = MessageBuilder.withPayload(msg)
                .build();
        source.output1().send(message);
    }

    /**
     * Send string with tag
     *
     * @param msg
     * @param tag
     */
    public void sendWithTags(String msg, String tag) {
        Message message = MessageBuilder.withPayload(msg)
                .setHeader(RocketMQHeaders.TAGS, tag)
                .build();
        source.output1().send(message);
    }

    /**
     * Sending objects
     *
     * @param msg
     * @param tag
     * @param <T>
     */
    public <T> void sendObject(T msg, String tag) {
        Message message = MessageBuilder.withPayload(msg)
                .setHeader(RocketMQHeaders.TAGS, tag)
                .setHeader(MessageHeaders.CONTENT_TYPE, MimeTypeUtils.APPLICATION_JSON)
                .build();
        source.output2().send(message);
    }
}

Write TestController.java controller for testing

package com.easy.arProduce;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping(value = "test")
public class TestController {
    @Autowired
    SenderService senderService;

    @RequestMapping(value = "/send", method = RequestMethod.GET)
    public String send(String msg) {
        senderService.send(msg);
        return "String message sent successfully!";
    }

    @RequestMapping(value = "/sendWithTags", method = RequestMethod.GET)
    public String sendWithTags(String msg) {
        senderService.sendWithTags(msg, "tagStr");
        return "belt tag String message sent successfully!";
    }

    @RequestMapping(value = "/sendObject", method = RequestMethod.GET)
    public String sendObject(int index) {
        senderService.sendObject(new Foo(index, "foo"), "tagObj");
        return "Object Object message sent successfully!";
    }
}

Create RocketMQ message consumer

Create Ali rocketmq consumer project with port 28082

pom.xml add dependency

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">

    <parent>
        <artifactId>cloud-alibaba</artifactId>
        <groupId>com.easy</groupId>
        <version>1.0.0</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>

    <artifactId>ali-rocketmq-consumer</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rocketmq</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

-Configure the Binding information of Input and validate it with @ EnableBinding annotation

application.yml configuration

server:
  port: 28082

spring:
  application:
    name: ali-rocketmq-consumer
  cloud:
    stream:
      rocketmq:
        binder:
          name-server: 127.0.0.1:9876 #rocketmq service address
        bindings:
          input1: {consumer.orderly: true}  #Is it sorted?
          input2: {consumer.tags: tagStr}   #Subscription string with tag value of tagStr
          input3: {consumer.tags: tagObj}   #Subscription string with tag value of tabObj
      bindings:
        input1: {destination: test-topic1, content-type: text/plain, group: test-group1, consumer.maxAttempts: 1}
        input2: {destination: test-topic1, content-type: application/plain, group: test-group2, consumer.maxAttempts: 1}
        input3: {destination: test-topic2, content-type: application/plain, group: test-group3, consumer.maxAttempts: 1}

management:
  endpoints:
    web:
      exposure:
        include: '*'
  endpoint:
    health:
      show-details: always

ArConsumerApplication.java

package com.easy.arConsumer;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.stream.annotation.EnableBinding;

@SpringBootApplication
@EnableBinding({MySource.class})
public class ArConsumerApplication {

    public static void main(String[] args) {
        SpringApplication.run(ArConsumerApplication.class, args);
    }
}

Message consumer service

MySource.java

package com.easy.arConsumer;

import org.springframework.cloud.stream.annotation.Input;
import org.springframework.messaging.SubscribableChannel;

public interface MySource {
    @Input("input1")
    SubscribableChannel input1();

    @Input("input2")
    SubscribableChannel input2();

    @Input("input3")
    SubscribableChannel input3();
}

ReceiveService.java

package com.easy.arConsumer;

import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.stereotype.Service;

@Service
@Slf4j
public class ReceiveService {

    @StreamListener("input1")
    public void receiveInput1(String receiveMsg) {
        log.info("input1 Message received:" + receiveMsg);
    }

    @StreamListener("input2")
    public void receiveInput2(String receiveMsg) {
        log.info("input2 Message received:" + receiveMsg);
    }

    @StreamListener("input3")
    public void receiveInput3(@Payload Foo foo) {
        log.info("input3 Message received:" + foo);
    }
}

Use example

Sample associated items

In this example, we create two project implementations

Ali RocketMQ producer: RocketMQ message service producer, service name: Ali RocketMQ producer, port: 28081
Ali RocketMQ consumer: RocketMQ message service consumer, service name: Ali RocketMQ producer, port: 28082

Run sample tests

First, start the Ali rocketmq producer service and Ali rocketmq consumer service

Visit the producer address of message service: http://localhost:28081/test/send?msg=yuntian

View service consumer console, output

2019-12-04 15:37:47.859  INFO 6356 --- [MessageThread_1] com.easy.arConsumer.ReceiveService       : input1 Message received: yuntian
2019-12-04 15:37:47.859  INFO 6356 --- [MessageThread_1] s.b.r.c.RocketMQListenerBindingContainer : consume C0A8096E200818B4AAC212CDA70E0014 cost: 1 ms

Indicates that string consumption is successfully consumed by input1

Visit the producer address of message service: http://localhost:28081/test/sendWithTags?msg=tagyuntian

View service consumer console, output

2019-12-04 15:38:09.586  INFO 6356 --- [MessageThread_1] com.easy.arConsumer.ReceiveService       : input2 Message received: tagyuntian
2019-12-04 15:38:09.592  INFO 6356 --- [MessageThread_1] com.easy.arConsumer.ReceiveService       : input1 Message received: tagyuntian
2019-12-04 15:38:09.592  INFO 6356 --- [MessageThread_1] s.b.r.c.RocketMQListenerBindingContainer : consume C0A8096E200818B4AAC212CDFCD30015 cost: 6 ms

It means that the string with tag is successfully consumed by input2 and input1, because input1 also subscribes to test-topic1, and we do not add tag filter. By default, it means that all messages are received, so the tagyuntian string can also be successfully received

Access the producer address of message service: http://localhost:28081/test/sendObject?index=1

View service consumer console, output

2019-12-04 15:41:15.285  INFO 6356 --- [MessageThread_1] com.easy.arConsumer.ReceiveService       : input3 Message received: Foo{id=1, bar='foo'}

It indicates that input3 has successfully received the object message with tag and tagObj, while input1 has no output message. This is because the message published by sendObject uses the test Topic2 message pipeline, so it will not be published to input1 and input2 subscribers

data

Topics: MySQL Spring Maven Apache Java

Programmer Think

Implementation of spring cloud Alibaba rocketmq asynchronous communication

introduce

RocketMQ features

Spring Cloud Stream

Setting up and deploying RocketMQ in Window

download

Start NameServer service

Start Broker service

Example

Create RocketMQ message producer

Create RocketMQ message consumer

Use example

Sample associated items

Run sample tests

data

Hot Topics