Message driven overview
What is Spring Cloud Stream? Officially, Spring Cloud Stream is a framework for building message driven microservices. Applications interact with binder objects in Spring Cloud Stream through inputs or outputs. We configure binding, and the binder object of Spring Cloud Stream is responsible for interacting with the message middleware. Therefore, we only need to figure out how to interact with Spring Cloud Stream to facilitate the use of message driven methods. By using Spring Integration to connect the message broker middleware to realize message event driven. Spring Cloud Stream provides personalized automatic configuration implementation for some vendors' message middleware products, citing the three core concepts of publish subscribe, consumption group and partition. Currently, only RabbitMQ and Kafka are supported.
One sentence explanation: shield the differences between the underlying message middleware, reduce the switching cost, and unify the message programming model
design idea
Standard MQ
| Information content is transmitted between producers / consumers through message media | Message |
| Messages must go through a specific channel | Message channel |
| How are messages in the message channel consumed? Who is responsible for sending and receiving messages | The interface SubscribableChannel is subscribed by the MessageHandler message processor |
Why Cloud Stream
Why can stream unify the underlying differences?
Without the concept of binder, when our SpringBoot application needs to directly interact with message oriented middleware,
Because the original intention of each message middleware is different, there will be great differences in their implementation details
By defining the binder as the middle layer, the isolation between application and message middleware details is perfectly realized.
By exposing the unified Channel channel to the application, the application does not need to consider a variety of different message middleware implementations.
By defining Binder as the middle layer, the isolation between application and message middleware details is realized.
Binder
Without the concept of binder, when our SpringBoot application wants to directly interact with the message middleware, due to the different original intention of each message middleware, there will be great differences in their implementation details. By defining the binder as the middle layer, it perfectly realizes the isolation between the application and the message middleware details. The further encapsulation of message oriented middleware by Stream can make the code layer insensitive to the middleware, and even dynamically switch the middleware (rabbitmq to kafka), which makes the development of micro services highly decoupled, and services can pay more attention to their own business processes
By defining Binder as the middle layer, the isolation between application and message middleware details is realized.
Binder can generate binding. Binding is used to bind producers and consumers of message containers. It has two types: INPUT and OUTPUT. INPUT corresponds to consumers and OUTPUT corresponds to producers.
The message communication mode in Stream follows the publish subscribe mode
Broadcast Topic RabbitMQ is Exchange In Kakfa, it is Topic
Spring Cloud Stream standard process routine
| Binder | It is very convenient to connect middleware to shield differences |
| Channel | Channel is an abstraction of Queue. In the message communication system, it is the medium for storage and forwarding. The Queue is configured through channel |
| Source and Sink | Simply understood, the reference object is the Spring Cloud Stream itself, Publishing a message from Stream is output, and receiving a message is input. |
Coding API and common annotations
Case description
RabbitMQ environment is OK
Three new sub modules are built in the project
Cloud stream rabbitmq provider 8801, as a producer, sends messages
Cloud stream rabbitmq consumer 8802 as a message receiving module
Cloud stream rabbitmq consumer 8803 as a message receiving module
Message driven producer
cloud-stream-rabbitmq-provider8801
POM
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<!--Basic configuration-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
yml
server:
port: 8801
spring:
application:
name: cloud-stream-provider
cloud:
stream:
binders: # Configure the service information of rabbitmq to be bound here;
defaultRabbit: # Represents the name of the definition, which is used for binding integration
type: rabbit # Message component type
environment: # Set the environment configuration related to rabbitmq
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
bindings: # Integration of services
output: # This name is the name of a channel
destination: studyExchange # Represents the Exchange name definition to use
content-type: application/json # Set the message type, json this time, and "text/plain" for text
binder: defaultRabbit # Set the specific settings of the message service to be bound
eureka:
client: # Configure Eureka registration on the client
service-url:
defaultZone: http://localhost:7001/eureka
instance:
lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (30 seconds by default)
lease-expiration-duration-in-seconds: 5 # If the interval of 5 seconds is exceeded now (the default is 90 seconds)
instance-id: send-8801.com # Displays the host name in the information list
prefer-ip-address: true # The access path becomes an IP address
Main startup class streamqmain8801
@SpringBootApplication
public class StreamMQMain8801
{
public static void main(String[] args)
{
SpringApplication.run(StreamMQMain8801.class,args);
}
}Business class
Send message interface
public interface IMessageProvider
{
public String send() ;
}Send message interface implementation class
@EnableBinding(Source.class) // It can be understood as the definition of a message sending pipeline
public class MessageProviderImpl implements IMessageProvider
{
@Resource
private MessageChannel output; // Message sending pipeline
@Override
public String send()
{
String serial = UUID.randomUUID().toString();
this.output.send(MessageBuilder.withPayload(serial).build()); // Create and send messages
System.out.println("***serial: "+serial);
return serial;
}
}
Controller
@RestController
public class SendMessageController
{
@Resource
private IMessageProvider messageProvider;
@GetMapping(value = "/sendMessage")
public String sendMessage()
{
return messageProvider.send();
}
}test
Start 7001eureka
Start rabbitmq rabbitmq-plugins enable rabbitmq_management http://localhost:15672/
Start 8801
visit http://localhost:8801/sendMessage
Message driven consumers
cloud-stream-rabbitmq-consumer8802
pom
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!--Basic configuration-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
yml
server:
port: 8802
spring:
application:
name: cloud-stream-consumer
cloud:
stream:
binders: # Configure the service information of rabbitmq to be bound here;
defaultRabbit: # Represents the name of the definition, which is used for binding integration
type: rabbit # Message component type
environment: # Set the environment configuration related to rabbitmq
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
bindings: # Integration of services
input: # This name is the name of a channel
destination: studyExchange # Represents the Exchange name definition to use
content-type: application/json # Set the message type. This time, it is an object json. If it is text, set "text/plain"
binder: defaultRabbit # Set the specific settings of the message service to be bound
eureka:
client: # Configure Eureka registration on the client
service-url:
defaultZone: http://localhost:7001/eureka
instance:
lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (30 seconds by default)
lease-expiration-duration-in-seconds: 5 # If the interval of 5 seconds is exceeded now (the default is 90 seconds)
instance-id: receive-8802.com # Displays the host name in the information list
prefer-ip-address: true # The access path becomes an IP address
Main startup class streamqmain8802
@SpringBootApplication
public class StreamMQMain8802
{
public static void main(String[] args)
{
SpringApplication.run(StreamMQMain8802.class,args);
}
}Business class
@Component
@EnableBinding(Sink.class)
public class ReceiveMessageListener
{
@Value("${server.port}")
private String serverPort;
@StreamListener(Sink.INPUT)
public void input(Message<String> message)
{
System.out.println("Consumer 1,------->Received message:" + message.getPayload()+"\t port: "+serverPort);
}
}Test 8801 sends 8802 receives messages
http://localhost:8801/sendMessage
Group consumption and persistence
According to 8802, clone will run 8803
start-up
RabbitMQ
seven thousand and one Service registration
eight thousand eight hundred and one Message production
eight thousand eight hundred and two Message consumption
eight thousand eight hundred and three Message consumption
There are two problems after running 1. There is the problem of repeated consumption 2. Message persistence
At present, 8802 / 8803 have been received at the same time, and there is a problem of repeated consumption
8802
8803
How to solve Grouping and persistence Properties group
Actual production cases
For example, in the following scenario, when we deploy the order system in a cluster, we will get the order information from RabbitMQ,
If an order is obtained by two services at the same time, it will cause data errors. We have to avoid this situation.
At this time, we can use the message grouping in the Stream to solve the problem.
Note that multiple consumers in the same group in the Stream are competitive, which can ensure that messages will only be consumed once by one of the applications.
Different groups can be fully consumed (repeated consumption),
Competition will occur in the same group, and only one of them can consume.
grouping
Principle: when microservice applications are placed in the same group, it can ensure that messages will only be consumed once by one of them.
Different groups can be consumed. There will be competition in the same group, and only one of them can be consumed.
8802 / 8803 become different groups, and the two groups are different group: A,B
8802 modify YML
server:
port: 8802
spring:
application:
name: cloud-stream-consumer
cloud:
stream:
binders: # configure the service information of rabbitmq to be bound here;
defaultRabbit: # indicates the name of the definition, which is used for binding integration
type: rabbit # message component type
Environment: # set the related environment configuration of rabbitmq
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
bindings: # service integration processing
input: # this name is the name of a channel, which will be explained when analyzing the specific source code
destination: studyExchange # indicates the Exchange name definition to use
Content type: application / json # sets the message type. This time, it is the object json. If it is text, set "text/plain"
binder: defaultRabbit # sets the specific settings of the message service to be bound
group: A
eureka:
Client: # client configures Eureka registration
service-url:
defaultZone: http://localhost:7001/eureka
instance:
Lease renewal interval in seconds: 2 # set the heartbeat interval (30 seconds by default)
Lease expiration duration in seconds: 5 # if the interval exceeds 5 seconds (90 seconds by default)
instance-id: receive-8802.com # Displays the host name in the information list
prefer-ip-address: true # The access path becomes an IP address
8803 modify YML
server:
port: 8803
spring:
application:
name: cloud-stream-consumer
cloud:
stream:
binders: # configure the service information of rabbitmq to be bound here;
defaultRabbit: # indicates the name of the definition, which is used for binding integration
type: rabbit # message component type
Environment: # set the related environment configuration of rabbitmq
spring:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
bindings: # service integration processing
input: # this name is the name of a channel, which will be explained when analyzing the specific source code
destination: studyExchange # indicates the Exchange name definition to use
Content type: application / json # sets the message type. This time, it is the object json. If it is text, set "text/plain"
binder: defaultRabbit # sets the specific settings of the message service to be bound
group: B
eureka:
Client: # client configures Eureka registration
service-url:
defaultZone: http://localhost:7001/eureka
instance:
Lease renewal interval in seconds: 2 # set the heartbeat interval (30 seconds by default)
Lease expiration duration in seconds: 5 # if the interval exceeds 5 seconds (90 seconds by default)
instance-id: receive-8803.com # Displays the host name in the information list
prefer-ip-address: true # The access path becomes an IP address
8802 / 8803 implements polling packets, with only one consumer at a time
Messages sent by 8801 module can only be received by one of 8802 or 8803, which avoids repeated consumption. 8802 / 8803 become the same group. Two groups are the same. Multiple micro service instances in the same group will only get one at a time
yml is changed to: group: A
Persistence
Through the above, the problem of repeated consumption is solved. Look at persistence, stop 8802 / 8803 and remove the group group:A of 8802. Group A of 8803 is not removed. 8801 sends 4 messages to rabbitmq first. Start 8802 first, there is no grouping attribute configuration, and there is no message printed in the background.
When 8803 is started again, there is a grouping attribute configuration, and messages on MQ are printed in the background
7001 module