1. What is middleware

What is middleware

Since the 1980s, Chinese enterprises have gradually carried out information construction. Due to the immature methods and systems and the changing market demand of enterprise business, an enterprise may run multiple different business systems at the same time. These systems may be based on different operating systems, different databases and heterogeneous network environment. The problem now is how to combine these information systems into an organic collaborative whole to truly realize enterprise cross platform and distributed applications. Middleware is the solution. It trades its complexity for the simplicity of enterprise applications.

Middleware is software between operating system and application program. Some people think it should be a part of operating system. When using middleware, people often integrate a group of middleware to form a platform (including development platform and operation platform), but there must be a communication middleware in this group of middleware, that is, middleware + platform + communication. This definition also limits that it can be called middleware only when used in distributed systems, At the same time, it can be distinguished from supporting software and using software

Why use message oriented middleware

Specifically, middleware shields the complexity of the underlying operating system, makes program developers face a simple and unified development environment, reduces the complexity of program design, focuses on their own business, and does not have to work repeatedly for the migration of programs on different system software, which greatly reduces the technical burden. Middleware brings to the application system, Not only the simplicity of development and the shortening of development cycle, but also the workload of system maintenance, operation and management, but also the investment of overall computer cost.

Middleware features

In order to solve the problem of distributed heterogeneity, the concept of middleware is proposed. Middleware is a general service between platforms (hardware and operating system) and applications. As shown in the figure below, these services have standard program interfaces and protocols. For different operating systems and hardware platforms, they can have a variety of implementations that meet the protocol specifications of interfaces.

[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-oL84NlWR-1633954870580)(image/1.jpg)]

It is also difficult to give a strict definition to middleware, but middleware should have the following characteristics:

(1) Meet the needs of a large number of applications

(2) Runs on a variety of hardware and OS platforms

(3) Support distributed computing and provide transparent interaction of applications or services across networks, hardware and OS platforms

(4) Support standard protocols

(5) Support standard interfaces

Due to the importance of standard interfaces for portability and standard protocols for interoperability, middleware has become the main part of many standardization work. For application software development, middleware is far more important than operating system and network services. The program interface provided by middleware defines a relatively stable high-level application environment, regardless of the underlying computer hardware and system software As long as the middleware is upgraded and the external interface definition of the middleware remains unchanged, the application software hardly needs any modification, so as to protect the significant investment of enterprises in the development and maintenance of application software.

To put it simply: middleware has a great feature. It is separated from the specific design objectives and has modules that provide universal independent functional requirements. This makes the middleware must be replaceable. If the middleware is irreplaceable in a system design, and there is no problem with the architecture and framework design, it is the middleware. It may be a middleware elsewhere in this system Inside is the engine.

When will middleware technology be used in the project

In the architecture and reconstruction of the project, we need to carefully consider and think about the use of any technology and architecture change, because the integration and change of any technology may increase personnel, technology and cost. Middleware technology is generally used more in some Internet companies or projects. If you are only a start-up company, it is recommended to use a single architecture At most, add a cache middleware. Don't blindly pursue new or so-called high performance. Behind the pursuit must be the business driver and project driver, because once the pursuit means that your learning cost, the company's personnel structure, server cost, maintenance and operation and maintenance cost will increase, so you need to choose and consider carefully.

However, as an open person, you must have the ability and thinking to learn middleware technology. Otherwise, it is easy to master estimation or mention it in the interview when the project develops to a stage, which will bring you a lot of trouble. In today's era, these technologies are not new. If you master and mine, the key is to spend your time and experience Through discussion and research.

2. Overview of middleware technology and architecture

Learn the ways and skills of Middleware

1. Understand the role of Middleware in the project architecture and the underlying implementation of each middleware
2. Some analogical life concepts can be used to understand middleware
3. Use some flow charts or brain maps to sort out the role of various middleware in the architecture
4. Try to use java technology to realize the principle of middleware
5. Calm down and think about the reasons for the design and use of Middleware in the project
6. If the corresponding alternative summary scheme is found
7. Try to write a blog to summarize the comparison and use scenarios of similar middleware technologies
8. Learn to view the source code of middleware, open source projects and blogs

What is message oriented middleware

In the actual project, most enterprise project development adopts the single architecture mode in the early morning

Monomer architecture

In enterprise development, most of the initial architectures adopt the mode of single architecture, and the typical feature of this architecture is to put all businesses and modules, source code, static resource files, etc. in one project. If a module is upgraded or iterated, the project will be recompiled and redeployed The problems with this dog are:

1. Coupling too high
2. Difficult to maintain
3. High cost of server
4. And the complexity of upgrading the architecture will also increase

In this way, there will be subsequent distributed architecture systems, as follows

Distributed architecture

What is a distributed system

More popular: a request is completed by multiple services (services or systems) on the server side

Different from the single architecture, the single architecture is that a request initiates a jvm scheduling Thread (specifically, tomcat Thread pool) to allocate a Thread thread Thread to process the request until it is released, while the distributed system is that multiple systems cooperate to complete a request, and the jvm and environment can be independent. If it is a metaphor in life, the single structure is like building a small house, which can be completed soon. If you want to build a bird's nest or a large building, you must coordinate and distribute all links. This purpose is also a problem to be deployed and considered when the project develops to the later stage. It is not difficult to see that the characteristics and problems of distributed architecture system are as follows:

Problems:

1. High learning cost and too many technology stacks
2. Increased operation and maintenance costs and server costs
3. The cost of personnel will also increase
4. The load of the project will also increase
5. The number of errors and fault tolerance will also multiply
6. The choice of occupied server port and communication cost is high
7. Security considerations and factors force you to choose RMI/MQ related server-side communication

Benefits:

1. The independence of the service system reduces the server resources occupied and the hardware cost occupied. To be exact, the service resources can be reasonably allocated
2. Source, no waste of server resources
3. Independent maintenance and deployment of the system, reduced coupling and pluggability
4. The choice of system architecture and technology stack can become flexible (rather than simply choosing java)
5. Flexible deployment will not cause the platform to be paralyzed or out of service due to deployment

3. Architecture of distributed system based on Message Oriented Middleware)

As can be seen from the above figure, the of message oriented middleware is

1. Use reliable message passing mechanism for direct communication between the system and the system
2. By providing message passing and message party mechanism, it can expand the communication between processes in the distributed system environment

Scenario of message oriented middleware application

1. Cross system data transfer
2. High concurrent traffic peak clipping
3. Concurrent and asynchronous processing of data
4. Big data analysis and transmission
5. Distributed transactions, such as when you have a data to migrate or too many concurrent requests,

For example, if you have 10 W concurrent requests to place orders, we can stack the order requests into the message queue before these orders are warehoused, so that they can be warehoused and executed stably and reliably

Common message oriented middleware

ActiveMQ, RabbitMQ, Kafka, RocketMQ, etc

The essence and design of Message Oriented Middleware

It is a technical service with the functions of receiving data, receiving requests, storing data, sending data and so on

MQ message queue: it is responsible for data transmission, reception, storage and transmission, so its performance is higher than that of ordinary services and technologies

Who will produce, store and consume messages?

The core component of Message Oriented Middleware

Message protocol
Message persistence mechanism
Message distribution policy
High availability and reliability of messages
Fault tolerance mechanism of message

Summary

In fact, whether you choose single architecture or distributed architecture is a stage of project development. At what stage, you should choose the appropriate architecture instead of blindly pursuing it. The final consequences and problems need to pay for themselves. But as a developer, learning and exploring new technologies makes it a problem that every program developer should keep and think about. When we can't change the society and the world, in order to live and survive, we must meet the needs of enterprises and the market, give full play to your value and learned talents, create value and realize ourselves

4. Message queuing protocol

What is an agreement

The so-called agreement means:

1. A set of conventions that the underlying operating system and application program of a computer follow when communicating. Only by following the common conventions and specifications can the system and the underlying operating system communicate with each other
2. Different from general network applications, it is mainly responsible for data acceptance and transmission, so its performance is relatively high
3. The protocol must strictly abide by the specifications for data format and data exchange between computers

Three elements of network protocol

1. Syntax: syntax is the structure and format of user data and control information, as well as the order in which data appears
2. Semantics: semantics is to explain the meaning of each part of the control information. It specifies what control information needs to be sent, what actions to be completed and what responses to be made
3. Timing: timing is a detailed description of the sequence of events

For example, I send a message to the queue in what data format, and then what is the meaning of each part, the actions executed after sending, the actions of consumer consumption messages, and the corresponding structure and feedback after consumption, and then process it according to the corresponding execution order. If you still don't understand the http request protocol that everyone contacts every day:

1. Syntax: http specifies the format of request message and response message
2. Semantics: a request initiated by a client is called a request (this is a definition, and you initiate a post/get request)
3. Timing: a request corresponds to a response (there must be a request before there is a response. This is timing)

The message middleware does not use http protocol, but the common message middleware protocols are: OpenWire, AMQP, MQTT, Kafka and OpenMessage protocol

Interview question: why does message oriented middleware not directly use http protocol

1. Because the http request message header and response message header are relatively complex, including cookies, data encryption and decryption, windowsill, response code and other additional functions. However, for a message, we do not need to be so complex or necessary. It is actually responsible for data transmission, storage and distribution. We must pursue high performance. Try to be concise and fast
2. In most cases, http is mostly short links. In the actual interaction process, a request to response is likely to be interrupted. After the interruption, persistence will not be executed, which will cause the loss of requests. This is not conducive to the business scenario of message middleware, because message middleware may be a long-term process of obtaining information. In case of problems and faults, it is necessary to persist the data or messages in order to ensure the highly reliable and stable operation of messages and data

AMQP protocol

AMQP: (full name: Advanced Message Queuing Protocol) is an Advanced Message Queuing Protocol. Jointly designed by JPMorgan Chase group and other companies. It is an application layer standard high-level message queuing protocol that provides unified messaging services. It is an open standard of application layer protocol. It is designed for message oriented middleware. The client and message middleware based on this protocol can deliver messages, which is not limited by different products and different development languages of the client / middleware. The implementation in Erlang is implemented by RabbitMQ et al

characteristic:

1. Distributed transaction support
2. Message persistence support
3. High performance and reliable message processing advantages

MQTT protocol

MQTT protocol (message queuing Telecommunication Transport) message queue is an open real-time communication protocol of IBM and an important part of the system architecture of the Internet of things

characteristic:

1. Light weight
2. Simple structure
3. Fast transmission, transaction not supported
4. No persistence design

Application scenario:

1. Applicable to limited computing power
2. Low bandwidth
3. Scenario of network instability

supporter:

OpenMessage protocol

It is an application development standard in the fields of Distributed Information Middleware and stream processing jointly founded by Alibaba, Yahoo, Didi travel, Stremalio and other companies in recent years

characteristic:

1. Simple structure
2. Fast parsing speed
3. Support transaction and persistence design

Kafka protocol

Kafka protocol is a binary protocol based on TCP/IP. The message is divided by length and consists of some basic data types

characteristic:

1. Simple structure
2. Fast parsing speed
3. No transaction support
4. Persistent design

Summary

Protocol: a protocol, commonly known as the specification and mechanism, built on the basis of tcp/ip protocol. Its main purpose is to enable the client (application java, go) to communicate and communicate. And this specification must have persistence, high availability and high reliability

5. Message queue persistence

Persistence

Simply put, it is to store the data on disk instead of in memory. It will disappear when the server is restarted and disconnected, so that the data can be saved permanently

Common persistence methods

Files and databases (rabbitmq), the rest are files

6. Message distribution strategy

Message distribution policy

MQ message queuing has the following roles

1. producer
2. Store message
3. consumer

After the producer generates the message, MQ stores it. How does the consumer get the message? Generally, there are two ways to obtain data: push or pull. A typical git has a push-pull mechanism. The http request we send is a typical process of pulling database data back. Message queue MQ is a push process, and these push mechanisms will use many business scenarios and have many corresponding push mechanism policies

Scenario analysis I

There are many systems and services. How do we know which system or server or system consumes this message? At this time, we need a distribution strategy. This requires a consumption strategy. Or call it the methodology of consumption

Scenario analysis II

In the process of sending messages, there may be exceptions, network jitter, faults, etc. because the messages cannot be consumed, such as the user placing an order, consuming MQ acceptance, and the order system failure, resulting in the user's payment failure. At this time, the message middleware must support the message retry mechanism strategy. That is, support: in case of problems and failures, messages can be retransmitted without loss
Mechanism and comparison of message distribution strategies

What is the high availability mechanism

The so-called high availability refers to the ability of the product to perform the specified function under the specified conditions and at the specified time or time

When the business volume increases, the request is too large. A message middleware server will touch the limit of hardware (CPU, memory and disk). A message server can no longer meet the needs of the business, so the message middleware must support cluster deployment to achieve the purpose of high availability

Cluster mode 1 - deployment of master slave shared data

Cluster mode 2 - Master Slave synchronous deployment mode

Explanation: messages written in this mode are also on the Master node, but the Master node will synchronize data to the slave node to form a replica, which is very similar to the zookeeper or redis Master-slave mechanism. In this way, the effect of load balancing can be achieved. If consumers have multiple nodes, they can go to different nodes for consumption, thinking that the copy and synchronization of messages will occupy a lot of bandwidth and network resources. It will be used in the later rabbitmq

Cluster mode 3 - Multi master cluster synchronous deployment mode

Explanation: it is not very different from the above, but it can be written to any node

Cluster mode 4 - Multi master cluster forwarding deployment mode

Explanation: if the data you insert is broker-1 China, the metadata information will store the relevant description of the data and the location (queue) where the records are stored. It synchronizes the description information, that is, the metadata information. If consumers consume in broker-2 and find that their nodes do not have corresponding information, they can query from the corresponding metadata information, and then return the corresponding message information. Scenarios: for example, buy train tickets or scalpers buy concert tickets. For example, the first scalpers have concert tickets that customers say they want to buy, But he didn't, but he went back to contact other scalpers to ask, and if so, he returned

Cluster mode 5: a combination of master slave and broker cluster

Explanation: the multi master and multi slave hot standby mechanism is implemented to complete the high availability of messages and the hot standby mechanism of data. When the production scale reaches a certain stage, this kind of use frequency is relatively high

What is a highly reliable mechanism

The so-called high reliability means that the system can operate continuously without failure. For example, a sudden system crash, error reporting, abnormality, etc. do not affect the normal operation of online business. The probability of error is very low, which is called high reliability

In high concurrency business scenarios, if the high reliability of the system cannot be guaranteed, the hidden dangers and losses will be very serious

How to ensure the reliability of middleware messages can be considered from two aspects:

1. Message transmission: ensure the correctness of data analysis between systems through protocols
2. Reliable message storage: ensure the reliability of messages through persistence

2, Introduction and installation

1. RabbitMQ introduction and installation

https://www.bilibili.com/video/BV1dX4y1V73G?p=27

01 overview

Brief overview:

RabbitMQ is an open source, implemented in accordance with AMQP protocol and written in Erlang language. It supports a variety of clients (Languages) for storing and forwarding messages in distributed systems. It has the characteristics of high availability, high scalability and ease of use

02 download RabbitMQ

1. Download address: https://www.rabbitmq.com/download.html
2. Environment preparation: CentOS7.x + /Erlang

RabbitMQ is developed in Erlang language, so the system environment must provide Erlang environment. The first step is to install Erlang

View system version number

Install Download

mkdir -p /usr/rabbitmq
ca /usr//rabbitmq
# Upload the installation package to the linux system
erlang-22.0.7-1.el7.x86_64.rpm
rabbitmq-server-3.7.18-1.el7.noarch.rpm

rpm -Uvh erlang-solutions-2.0-1.noarch.rpm
yum install -y erlang
erl -v

04 installing socat

Install Download

yum install -y socat

05 installing rabbitmq

Install Download

rpm -Uvh rabbitmq-server-3.7.18-1.el7.noarch.rpm
yum install rabbitmq-server -y

Start service

# Start service
systemctl start rabbitmq-server
# View the service status, as shown in the figure
systemctl status rabbitmq-server.service
# Startup self startup
systemctl enable rabbitmq-server
# Out of Service
systemctl stop rabbitmq-server

2. RabbitMQWeb management interface and authorization operation

01 RabbitMQ management interface

By default, the web client plug-in is not installed. It needs to be installed before it can take effect

rabbitmq-plugins enable rabbitmq_management

Note: rabbitmq has a default account and password: guest can only be accessed under localhost by default, so you need to add a remote login user

After installation, restart the service

systemctl restart rabbitmq-server

Remember to open port 15672 in the security group of the corresponding server (Alibaba cloud, Tencent cloud, etc.)

Access in browser

# 10. Turn off the firewall service
systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
systemctl stop firewalld   
# 11. Access the web management interface
http://10.15.0.8:15672/

02 authorized account and password

New user

rabbitmqctl add_user admin admin

Set user assigned operation permissions

rabbitmqctl set_user_tags admin administrator

User level:

1. administrator: you can log in to the console, view all information, and manage rabbitmq
2. monitoring: the monitor logs in to the console to view all information
3. policymaker: the policy maker logs in to the console and specifies the policy
4. Management common administrator login console

Add resource permissions for users

rabbitmqctl set_permissions -p / admin ".*"".*"".*"

Web login succeeded

03 summary:

3. Docker installation of rabbitmq

01 Dokcer installing RabbitMQ

Get rabbit image

docker pull rabbitmq:management

Create and run containers

docker run -id --name=myrabbit -p 15672:15672 rabbitmq:management
--hostname: Specifies the container host name
--name:Specify container name
-p: take mq Port number mapped to local
 Or set the user and password at runtime

start-up

Visit the web page successfully!

4. Role classification of rabbitmq

3, Introductory case

1. Simple mode
2. Work mode
3. Publish subscribe mode
4. Routing mode
5. Topic mode
6. Parameter mode

1. RabbitMQ introduction case - Simple mode

https://www.bilibili.com/video/BV1dX4y1V73G?p=44 Implementation steps

1. jdk1.8
2. Build a maven project
3. Import maven dependency of rabbitmq
4. Start rabbitmq server service
5. Define producer
6. Define consumer
7. Observe the process of the message in the rabbitmq server service

01 build a maven project

02 import dependency

java Native dependency

<dependency>
    <groupId>com.rabbitmq</groupId>
    <artifactId>amqp-client</artifactId>
    <version>5.10.0</version>
</dependency>

03 first model

In the model above, there are the following concepts:

1. The producer is the program that sends the message
2. Consumer: the receiver of the message will always wait for the message to arrive.
3. Message queue: the red part in the figure. Similar to a mailbox, messages can be cached; The producer delivers the message to it, and the consumer takes the message out of it.

producer

//Simple mode
public class Producer{
    //1. Create a connection factory
    ConnectionFactory connectionFactory = new ConnectionFactory();
    connectionFactory.setHost("10.15.0.9");
    connectionFactory.setPort(5672);
    connectionFactory.setUsername("admin");
    connectionFactory.setPassword("admin");
    connectionFactory.setVirtualHost("/");
    Connection connection = connectionFactory.newConnection("producer");
    //2. Create channel
    Channel channel = connection.createChannel();
    //3. Create switches, declare queues, bind relationships, route key s, send and receive messages
    /*Parameter 1: whether to persist. Will non persistent messages be saved? The disk will be saved, but it will be lost when the server is restarted
      Parameter 2: exclusive queue 
      Parameter 3: whether to delete automatically. Whether to delete the queue automatically after the last consumer message is completed
  	  Parameter 4: carry subsidiary attributes
    */
    String queueName = "queue1";
    channel.queueDeclare(queueName,false,false,false,null);
    //4. Send message to queue
    /*Parameter 1: switch
      Parameter 2: queue, routing key
      Parameter 3: status control of message
  	  Parameter 4: message subject
    */
    //Interview question: can there be queues without switches? Impossible. Although no switch is specified, there must be a default switch
    String message = "Hello";
    channel.basicPublish("",message, null,message.getBytes());
    //5. Close
    channel.close();
    connection.close();
}

consumer

//Simple mode
public class Consumer{
    //1. Create a connection factory
    ConnectionFactory connectionFactory = new ConnectionFactory();
    connectionFactory.setHost("10.15.0.9");
    connectionFactory.setPort(5672);
    connectionFactory.setUsername("admin");
    connectionFactory.setPassword("admin");
    connectionFactory.setVirtualHost("/");
    Connection connection = connectionFactory.newConnection("producer");
    //2. Create channel
    Channel channel = connection.createChannel();
	//3. Accepted content
    channel.basicConsume("queue1",true,new DefaultConsumer(){
        public void handle(String consumerTag, Delivery message) throws IOException {
          System.out.println(new String("The message is" + new String(meassage.getBody()),"UTF-8"));
        },new CancelCallback(){
            public void handle(String consumerTag) throws IOException {
                System.out.println("Acceptance failed");
        }
      });
    //4. Close
    channel.close();
    connection.close();
}

4. RabbitMQ entry case - fanout mode

01 publish subscribe mode of rabbitmq

graphic

1. web operation viewing video
2. Type: fanout
3. Features: Fanout - publish and subscribe mode is a broadcast mechanism. It is a mode without routing key

producer

//Simple mode
public class Producer{
    //1. Create a connection factory
    ConnectionFactory connectionFactory = new ConnectionFactory();
    connectionFactory.setHost("10.15.0.9");
    connectionFactory.setPort(5672);
    connectionFactory.setUsername("admin");
    connectionFactory.setPassword("admin");
    connectionFactory.setVirtualHost("/");
    Connection connection = connectionFactory.newConnection("producer");
    //2. Create channel
    Channel channel = connection.createChannel();
    //3. Create switches, declare queues, bind relationships, route key s, send and receive messages
    /*Parameter 1: whether to persist. Will non persistent messages be saved? The disk will be saved, but it will be lost when the server is restarted
      Parameter 2: exclusive queue 
      Parameter 3: whether to delete automatically. Whether to delete the queue automatically after the last consumer message is completed
  	  Parameter 4: carry subsidiary attributes
    */
    String queueName = "queue1";
    channel.queueDeclare(queueName,false,false,false,null);
    //4. Send message to queue
    /*Parameter 1: switch
      Parameter 2: queue, routing key
      Parameter 3: status control of message
  	  Parameter 4: message subject
    */
    //Interview question: can there be queues without switches? Impossible. Although no switch is specified, there must be a default switch
    String message = "Hello";
    //5. Prepare the switch
    String exchangeName = "fanout-exchange";
    //6. Define routing key
    String routeKey = "";
    //7. Specify the type of switch
    String type = "fanout";
    channel.basicPublish(exchangeName,routeKey, null,message.getBytes());
    //8. Close
    channel.close();
    connection.close();
}

consumer

The code is the same, just start the test with thread!

Switches and queues are not bound through code here, but through the visual interface!

5. RabbitMQ entry case - Direct mode

//6. Define routing key
String routeKey = "email";
//7. Specify the type of switch
String type = "direct";
channel.basicPublish(exchangeName,routeKey, null,message.getBytes());

6. RabbitMQ introduction case - Topic mode

//6. Define routing key
String routeKey = "com.order.test.xxx";
//7. Specify the type of switch
String type = "direct";
channel.basicPublish(exchangeName,routeKey, null,message.getBytes());

Code creation and binding

//5. Prepare the switch
String exchangeName = "direct_message_exchange";
String exchangeType = "direct";
//If you use the interface to bind the relationship between queue and exchange first, the code does not need to write these declaration codes, which can make the code more concise
//If we declare it in code, we need to learn
//6. Declare that the so-called persistence of the switch refers to whether the switch will be lost with the server restart
channel.exchangeDeclare(exchangeName,exchangeType,true);

//7. Claim queue
channel.queueDeclare("queue5",true,false,false,null);
channel.queueDeclare("queue6",true,false,false,null);
channel.queueDeclare("queue7",true,false,false,null);

//8. Relationship between binding queue and switch
channel.queueBind("queue5",exchangeName,"order");
channel.queueBind("queue6",exchangeName,"order");
channel.queueBind("queue7",exchangeName,"course");

channel.basicPublish(exchangeName,course, null,message.getBytes());

7. RabbitMQ entry case - Work mode

01 Work mode polling mode (round robin)

When there are multiple consumers, which consumer will consume our news, and how can we balance the amount of consumer consumption information?

There are two main modes:

1. Distribution in polling mode: one for each consumer, distributed equally
2. Fair distribution: fair distribution is carried out according to the consumption capacity of consumers, with more fast processing and less slow processing; distribution according to work

producer

Just like simple mode!

consumer

Create two of the same!

02 Work mode fair distribution mode

producer

Just like simple mode!

consumer

//Simple mode
public class Consumer{
	//3. Accepted content
    //Indicators are defined
    channel.basicQos(1);
    channel.basicConsume("queue1",false,new DefaultConsumer(){
        public void handle(String consumerTag, Delivery message) throws IOException {
          System.out.println(new String("The message is" + new String(meassage.getBody()),"UTF-8"));
          //Change to manual response
          channel.basicAck(delivery.getEnvelope().getDeliveryTag(),false);
        },new CancelCallback(){
            public void handle(String consumerTag) throws IOException {
                System.out.println("Acceptance failed");
        }
      });
    //4. Close
    channel.close();
    connection.close();
}

Create two of the same!

8. RabbitMQ usage scenario

01 decoupling, peak shaving, asynchronous

Synchronous asynchronous problem (serial)

Serial mode: after successfully writing the order information into the database, send the registration email and then send the registration SMS. After the above three tasks are completed, return to the client

public void makeOrder(){
    //1. Send order
    //2. Send SMS service
    //3. Send email service
    //4. Send app service
}

Parallel asynchronous thread pool

Parallel mode: after the order information is successfully written into the database, send the registration email and send the registration SMS at the same time. After the above three tasks are completed, return to the client. The difference from serial is that the parallel method can improve the processing time

public void test(){
    //asynchronous
    theadpool.submit(new Callable<Object>{
        //1. Send SMS service
    })
    //asynchronous
    theadpool.submit(new Callable<Object>{
        //2.
    })
    //asynchronous
    theadpool.submit(new Callable<Object>{
        //3.
    })
    //asynchronous
    theadpool.submit(new Callable<Object>{
        //4.
    })
}

Existing problems

1. High coupling
2. You need to write your own thread pool. The maintenance cost is too high
3. Messages may be lost. You need to make message compensation yourself
4. How to ensure the reliability of the message? Write it yourself
5. If the server cannot be hosted, you need to write high availability

Asynchronous message queuing

Benefits:

1. Fully decoupled, bridging with MQ
2. There is an independent thread pool and running model
3. If a message occurs, it may be lost. MQ has persistence function
4. How to ensure the reliability of messages, dead letter queue and message transfer, etc
5. If the server cannot be hosted, you need to write high availability, and the HA image model is high availability

According to the above agreement, the user's response time is equivalent to the time when the order information is written to the database, that is, 50 milliseconds. Register e-mail, send SMS, write to the message queue and return directly. Therefore, the speed of writing to the message queue is very fast and can be basically ignored. Therefore, the user's response time may be 50 milliseconds. Therefore, after the architecture is changed, the throughput of the system is increased to 20QPS per second. It is three times higher than serial and two times higher than parallel

02 high cohesion, low coupling

Benefits:

1. Fully decoupled, bridging with MQ
2. There is an independent thread pool and running model
3. If a message occurs, it may be lost. MQ has persistence function
4. How to ensure the reliability of messages, dead letter queue and message transfer, etc
5. If the server cannot be hosted, you need to write high availability, and the HA image model is high availability

According to the above agreement, the user's response time is equivalent to the time when the order information is written to the database, that is, 50 milliseconds. Register e-mail, send SMS, write to the message queue and return directly. Therefore, the speed of writing to the message queue is very fast and can be basically ignored. Therefore, the user's response time may be 50 milliseconds. Therefore, after the architecture is changed, the throughput of the system is increased to 20QPS per second. It is three times higher than serial and two times higher than parallel

4, Springboot case

1. Fanout mode

https://www.bilibili.com/video/BV1dX4y1V73G?p=44

producer

application.yml

# Service port
server:
  port: 8080
# Configure rabbitmq service
spring:
	rabbitmq:
		username: admin
		password: admin
		virtual-host: /
		host: 127.0.0.1
		port: 5672

OrderService.java

public class OrderService{
    @Autowired
    private RabbitTemplate rabbitTemplate;
    //Simulated user orders
    public void makeOrder(String userid,String productid,int num){
        //1. Query whether the inventory is sufficient according to the commodity id
        //2. Save order
        String orderId = UUID.randomUUID().toString();
        sout("Order production succeeded:"+orderId);
        //3. Complete message distribution through MQ
        //Parameter 1: switch parameter 2: routing key/queue queue name parameter 3: message content
        String exchangeName = "fanout_order_exchange";
        String routingKey = "";
        rabbitTemplate.convertAndSend(exchangeName,routingKey,orderId);
    }
}

consumer

application.yml

# Service port
server:
  port: 8080
# Configure rabbitmq service
spring:
	rabbitmq:
		username: admin
		password: admin
		virtual-host: /
		host: 127.0.0.1
		port: 5672

RabbitMqConfiguration.java

@Configuration
public class RabbitMqConfiguration{
    //1. Declare and register the switch in fanout mode
    @Bean
    public FanoutExchange fanoutExchange(){
        return new FanoutExchange("fanout_order_exchange",true,false);
    }
    //2. Claim queue
    @Bean
    public Queue smsQueue(){
        return new Queue("sms.fanout.queue",true);
    }
    @Bean
    public Queue duanxinQueue(){
        return new Queue("duanxin.fanout.queue",true);
    }
    @Bean
    public Queue emailQueue(){
        return new Queue("email.fanout.queue",true);
    }
    //3. Complete the binding relationship
    @Bean
    public Binding smsBingding(){
        return BindingBuilder.bin(smsQueue()).to(fanoutExchange());
    }
    @Bean
    public Binding duanxinBingding(){
        return BindingBuilder.bin(duanxinQueue()).to(fanoutExchange());
    }
    @Bean
    public Binding emailBingding(){
        return BindingBuilder.bin(emailQueue()).to(fanoutExchange());
    }
}

FanoutSmsConsumer.java

@Component
@RabbitListener(queue = {"sms.direct.queue"})
public class FanoutSmsConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("sms The order information received is:"+message);
    }
}

FanoutDuanxinConsumer.java

@Component
@RabbitListener(queue = {"duanxin.direct.queue"})
public class FanoutDuanxinConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("duanxin The order information received is:"+message);
    }
}

FanoutEmailConsumer.java

@Component
@RabbitListener(queue = {"duanxin.direct.queue"})
public class FanoutEmailConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("email The order information received is:"+message);
    }
}

2. Direct mode

producer

OrderService.java

public class OrderService{
    @Autowired
    private RabbitTemplate rabbitTemplate;
    //Simulated user orders
    public void makeOrder(String userid,String productid,int num){
        //1. Query whether the inventory is sufficient according to the commodity id
        //2. Save order
        String orderId = UUID.randomUUID().toString();
        sout("Order production succeeded:"+orderId);
        //3. Complete message distribution through MQ
        //Parameter 1: switch parameter 2: routing key/queue queue name parameter 3: message content
        String exchangeName = "direct_order_exchange";
        String routingKey = "";
        rabbitTemplate.convertAndSend(exchangeName,"email",orderId);
        rabbitTemplate.convertAndSend(exchangeName,"duanxin",orderId);
    }
}

consumer

RabbitMqConfiguration.java

@Configuration
public class RabbitMqConfiguration{
    //1. Declare and register the switch in fanout mode
    @Bean
    public DirectExchange directExchange(){
        return new DirectExchange("direct_order_exchange",true,false);
    }
    //2. Claim queue
    @Bean
    public Queue smsQueue(){
        return new Queue("sms.direct.queue",true);
    }
    @Bean
    public Queue duanxinQueue(){
        return new Queue("duanxin.direct.queue",true);
    }
    @Bean
    public Queue emailQueue(){
        return new Queue("email.direct.queue",true);
    }
    //3. Complete the binding relationship
    @Bean
    public Binding smsBingding(){
        return BindingBuilder.bin(smsQueue()).to(fanoutExchange()).with("sms");
    }
    @Bean
    public Binding duanxinBingding(){
        return BindingBuilder.bin(duanxinQueue()).to(fanoutExchange()).with("duanxin");
    }
    @Bean
    public Binding emailBingding(){
        return BindingBuilder.bin(emailQueue()).to(fanoutExchange()).with("email");
    }
}

3. Topic mode

producer

public class OrderService{
    @Autowired
    private RabbitTemplate rabbitTemplate;
    //Simulated user orders
    public void makeOrder(String userid,String productid,int num){
        //1. Query whether the inventory is sufficient according to the commodity id
        //2. Save order
        String orderId = UUID.randomUUID().toString();
        sout("Order production succeeded:"+orderId);
        //3. Complete message distribution through MQ
        //Parameter 1: switch parameter 2: routing key/queue queue name parameter 3: message content
        String exchangeName = "direct_order_exchange";
        String routingKey = "com.duanxin";
        rabbitTemplate.convertAndSend(exchangeName,routingKey,orderId);
    }
}

Consumer (using notes)

FanoutSmsConsumer.java

@Component
@RabbitListener(bindings = @QueueBinding(
	value = @Queue(value = "sms.topic.queue",durable = "true",antoDelete = "false"),
    exchange = @Exchange(value = "topic_order_exchange",type = "ExchangeTypes.TOPIC")
    key = "#.sms.#"
))
public class TopicSmsConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("sms The order information received is:"+message);
    }
}

FanoutDuanxinConsumer.java

@Component
@RabbitListener(bindings = @QueueBinding(
	value = @Queue(value = "duanxin.topic.queue",durable = "true",antoDelete = "false"),
    exchange = @Exchange(value = "topic_order_exchange",type = "ExchangeTypes.TOPIC")
    key = "#.duanxin.#"
))
public classTopicDuanxinConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("duanxin The order information received is:"+message);
    }
}

FanoutEmailConsumer.java

@Component
@RabbitListener(bindings = @QueueBinding(
	value = @Queue(value = "email.topic.queue",durable = "true",antoDelete = "false"),
    exchange = @Exchange(value = "topic_order_exchange",type = "ExchangeTypes.TOPIC")
    key = "#.email.#"
))
public class TopicEmailConsumer{
    @RabbitHandler
    public void reviceMessage(String message){
        sout("email The order information received is:"+message);
    }
}

5, RabbitMQ advanced

##1. Expiration time TTL

https://www.bilibili.com/video/BV1dX4y1V73G?p=44

summary

The expiration time TTL indicates that the expected time can be set for the message and can be received by the consumer within this time; After that, the message will be deleted automatically. RabbitMQ can set TTL for messages and queues. At present, there are two methods to set TTL

1. The first method is to set the queue property so that all messages in the queue have the same expiration time
2. The second method is to set the message separately, and the TTL of each message can be different

If the above two methods are used at the same time, the expiration time of the message shall be subject to the value with the smaller TTL. Once the lifetime of the message in the queue exceeds the set TTL value, it is called dead message. It is delivered to the dead letter queue, and the consumer will no longer receive the message

Set queue TTL

RabbitMqConfiguration.java

@Configuration
public class TTLRabbitMQConfiguration{
    //1. Declare and register the switch in direct mode
    @Bean
    public DirectExchange ttldirectExchange(){
        return new DirectExchange("ttl_direct_exchange",true,false);}
    //2. Expiration time of the queue
    @Bean
    public Queue directttlQueue(){
        //Set expiration time
        Map<String,Object> args = new HashMap<>();
        args.put("x-message-ttl",5000);//This must be of type int
        return new Queue("ttl.direct.queue",true,false,false,args);}
    
    @Bean
    public Binding ttlBingding(){
        return BindingBuilder.bin(directttlQueue()).to(ttldirectExchange()).with("ttl");
    }
}

Set message TTL

public class OrderService{
    @Autowired
    private RabbitTemplate rabbitTemplate;
    //Simulated user orders
    public void makeOrder(String userid,String productid,int num){
        //1. Query whether the inventory is sufficient according to the commodity id
        //2. Save order
        String orderId = UUID.randomUUID().toString();
        sout("Order production succeeded:"+orderId);
        //3. Complete message distribution through MQ
        //Parameter 1: switch parameter 2: routing key/queue queue name parameter 3: message content
        String exchangeName = "ttl_order_exchange";
        String routingKey = "ttlmessage";
        //Set expiration time for messages
        MessagePostProcessor messagePostProcessor = new MessagePostProcessor(){
            public Message postProcessMessage(Message message){
                //Here is the string
                message.getMessageProperties().setExpiration("5000");
                message.getMessageProperties().setContentEncoding("UTF-8");
                return message;
            }
        }
        rabbitTemplate.convertAndSend(exchangeName,routingKey,orderId,messagePostProcessor);
    }
}

RabbitMqConfiguration.java

@Configuration
public class TTLRabbitMQConfiguration{
    //1. Declare and register the switch in direct mode
    @Bean
    public DirectExchange ttldirectExchange(){
        return new DirectExchange("ttl_direct_exchange",true,false);}
    //2. Expiration time of the queue
    @Bean
    public Queue directttlQueue(){
        //Set expiration time
        Map<String,Object> args = new HashMap<>();
        args.put("x-message-ttl",5000);//This must be of type int
        return new Queue("ttl.direct.queue",true,false,false,args);}
    @Bean
    public Queue directttlMessageQueue(){
        return new Queue("ttlMessage.direct.queue",true,false,false,args);}
    
    @Bean
    public Binding ttlBingding(){
        return BindingBuilder.bin(directttlMessageQueue()).to(ttldirectExchange()).with("ttlmessage");
    }
}

2. Dead letter queue

summary

DLX, fully known as dead letter exchange, can be called dead letter switch or dead letter mailbox. When a message becomes dead letter in another queue, it can be re sent to another switch, which is DLX. The queue bound to DLX is called dead letter queue. The message may become dead letter due to the following reasons:

1. Message rejected
2. Message expiration
3. The queue has reached its maximum length

DLX is also a normal switch. It is no different from a general switch. It can be specified on any queue. In fact, it is to set the properties of a queue. When there is dead letter in this queue, Rabbitmq will automatically republish the message to the set DLX, and then route it to another queue, that is, dead letter queue.

To use dead letter queue, you only need to set the queue parameter x-dead-letter-exchange to specify the switch when defining the queue

code

DeadRabbitMqConfiguration.java

@Configuration
public class DeadRabbitMqConfiguration{
    //1. Declare and register the switch in direct mode
    @Bean
    public DirectExchange deadDirect(){
        return new DirectExchange("dead_direct_exchange",true,false);}
    //2. Expiration time of the queue
    @Bean
    public Queue deadQueue(){
        return new Queue("dead.direct.queue",true);}
    @Bean
    public Binding deadbinds(){
        return BindingBuilder.bind(deadDirect()).to(deadQueue()).with("dead");
    }
}

RabbitMqConfiguration.java

@Configuration
public class TTLRabbitMQConfiguration{
    //1. Declare and register the switch in direct mode
    @Bean
    public DirectExchange ttldirectExchange(){
        return new DirectExchange("ttl_direct_exchange",true,false);}
    //2. Expiration time of the queue
    @Bean
    public Queue directttlQueue(){
        //Set expiration time
        Map<String,Object> args = new HashMap<>();
        //args.put("x-max-length",5);
        args.put("x-message-ttl",5000);//This must be of type int
        args.put("x-dead-letter-exchange","dead_direct_exchange");
        args.put("x-dead-letter-routing-key","dead");//fanout does not need to be configured
        return new Queue("ttl.direct.queue",true,false,false,args);}
    @Bean
    public Queue directttlMessageQueue(){
        return new Queue("ttlMessage.direct.queue",true,false,false,args);}
    
    @Bean
    public Binding ttlBingding(){
        return BindingBuilder.bin(directttlMessageQueue()).to(ttldirectExchange()).with("ttlmessage");
    }
}