catalogue
1.2 what are the advantages of MQ
1.3 different MQ characteristics
3 message model supported by rabbitmq
3.3 the first model (direct connection)
3.4 encapsulation of connection tools in rabbitmq
3.6 the second model (work queue)
4. The consumer creates two: / / the difference is whether to bring the following thread sleep code
3.7. The third model (fanout | broadcast)
1. In broadcast mode, the message sending process is as follows:
1. Routing subscription model - Direct
3.9. Fifth model (Topics) dynamic routing
2. The producer is currently user save
1.MQ
1.1 definitions
MQ (message queue): translated into message queue. Through the typical producer and consumer model, producers continuously produce messages to the message queue, and consumers continuously obtain messages from the queue. Because the production and consumption of messages are asynchronous, and only care about the sending and receiving of messages, without the intrusion of business logic, it is easy to realize the decoupling between systems. The alias is message middleware, which uses efficient and reliable message passing mechanism for platform independent data exchange, and integrates distributed systems based on data communication.
1.2 what are the advantages of MQ
There are many mainstream message middleware in the market today, such as the old ActiveMQ and RabbitMQ, the hot Kafka, and RocketMQ independently developed by Alibaba.
1.3 different MQ characteristics
1.ActiveMQ ActiveMQ yes Apache Product, the most popular and powerful open source message bus. It is a fully supported JMS Standardized message oriented middleware. abundant API,Multiple cluster architecture modes ActiveMQ It has become an old message middleware in the industry,It is very popular in small and medium-sized enterprises!Performance is not as good as others. Throughput is not high. 2.Kafka Kafka yes LinkedIn Open source distributed Publishing-Subscription message system, which currently belongs to Apache Top projects. Kafka The main feature is based on Pu11 To process message consumption and pursue high throughput, the initial purpose is to collect and transmit logs. 0.8 The version supports replication, does not support transactions, and has no strict requirements on message duplication, loss and error. It is suitable for the data collection business of Internet services that generate a large amount of data. For big data real-time processing 3.RocketMQ RocketMQ It is Alibaba's open source message middleware. It is pure Java It has the characteristics of high throughput, high availability and suitable for large-scale distributed system applications. RocketMQ Ideas originate from Kafka,But it's not Kafka of-individual Copy,It optimizes the reliable transmission and transaction of messages. At present, it is widely used in transaction, recharge, stream computing, message push, log stream processing bing1og Distribution and other scenarios. It is used for double eleven, but Alibaba cloud bought it RocketMQ To support transactions. 4.RabbitMQ RabbitMQ Is to use Erlang The open source message queue system developed by language (so when dealing with concurrency, the performance is very good), based on AMQP Protocol. AMQP The main feature of is message oriented, queue oriented and routing oriented(Includes point-to-point and publishing/subscribe),Reliability and safety. AMQP The protocol is more used in the scenarios with high requirements for data consistency, stability and reliability in the enterprise system (the highest of the four types will not lose any data), and the requirements for performance and throughput are second (lost to the second) Kafka). And spring Seamless butt joint of frame
Conclusion: RabbitMQ is more reliable than Kafka. Kafka is more suitable for IO high throughput processing. It is generally used in big data log processing or scenarios with slightly lower requirements for real-time (a small amount of delay) and reliability (a small amount of data loss), such as ELK log collection.
2 RabbitMQ
Based on AMQP protocol and erlang language development, it is the most widely deployed open source message middleware and one of the most popular open source message middleware.
Official website: https://www.rabbitmq.com/
Official tutorial: https://www.rabbitmq.com/getstarted.html
AMQP protocol
AMQP (advanced message queuing protocol) was proposed in 2003 and was first used to solve the problem of message passing interaction between different financial platforms. As the name suggests, AMQP is a protocol, more precisely a binary wireless protocol. This is the essential difference between AMQP and JMS. AMQP does not limit from the API layer, but directly defines the data format of network exchange. This makes the provider nature of AMQP cross platform. The following is the AMQP protocol model:
The producer sends the message to the switch. Switches and queues are bound one by one, that is, point-to-point. Switches can route to other queues.
3 message model supported by rabbitmq
3.1 official website
https://www.rabbitmq.com/getstarted.html
3.2 introducing dependencies
<!--introduce rabbitmq Related dependencies of-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.7.3</version>
</dependency>Why do you need to delete the following line when writing test code? Because this is the scope of action,
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<!-- <scope>test</scope>-->
</dependency>3.3 the first model (direct connection)
In the model above, there are the following concepts:
P: The producer is the program that sends the message
C: Consumer: the receiver of the message will - wait for the message to arrive.
Queue: message queue, the red part in the figure. Similar to a mailbox, messages can be cached; The producer delivers the message to it, and the consumer takes the message out of it.
1. Developing producers
public class Provider {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
// Create a connection to the server
ConnectionFactory factory = new ConnectionFactory();
// Connect rabbitmq host
factory.setHost("192.168.231.141");
// Set port number
factory.setPort(5672);
// Set which virtual host to connect to
factory.setVirtualHost("/ems");
// Set the user name and password to access the virtual host
factory.setUsername("ems");
factory.setPassword("123456");
// Get connection object
try (Connection connection = factory.newConnection();
// Gets the channel object in the connection
Channel channel = connection.createChannel()) {
// Message queue corresponding to channel binding
// Parameter 1 Queue name if the queue does not exist, it is automatically created
// 2. durable is used to define whether the characteristics of the queue should be persisted or not
// 3. Exclusive queue
// 4. Whether to delete the queue automatically after consumption is completed. If true, delete it automatically
// 5. Additional parameters
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "Hello World!";
// Release news
// 1. The switch name is empty because there is no switch
// 2. Queue name
// 3. Properties when publishing messages - additional settings for delivering messages
// 4. The specific content of the published message shall be an array of bytes,
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
System.out.println(" [x] Sent '" + message + "'");
}
}
}
There is no obvious closing code here because the official website says:
2. Develop consumers
Consumed the news
The reason why you don't need to close the code here is that you want him to listen all the time and get the message directly when there is a message:
3.4 encapsulation of connection tools in rabbitmq
public abstract class RabbitMQUtils {
// Heavyweight resources. You don't want to create them one at a time. Instead, you want to create it when the class is loaded.
private static ConnectionFactory factory = new ConnectionFactory();
// Static code block, which is executed only once when the class is loaded
static {
// Connect rabbitmq host
factory.setHost("192.168.231.141");
// Set port number
factory.setPort(5672);
// Set which virtual host to connect to
factory.setVirtualHost("/ems");
// Set the user name and password to access the virtual host
factory.setUsername("ems");
factory.setPassword("123456");
}
public static Connection getConnection() {
try {
// Get connection object
Connection connection = factory.newConnection();
return connection;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}3.5 details
Queue binding channel
// Message queue corresponding to channel binding
// Parameter 1 Queue name if the queue does not exist, it is automatically created
// 2. durable is used to define whether the characteristics of the queue should be persisted. true persistence refers to saving the queue in the disk. When rabbitmq service is restarted, the queue and messages in the queue will be lost
// 3. Exclusive queue
// 4. Whether to delete the queue automatically after consumption is completed. If true, delete it automatically
// 5. Additional parameters
channel.queueDeclare("A", false, false, false, null);However, if you are not bound, you must send messages here. It is determined by the following code:
String message = "Hello World!";
// Release news
// 1. The switch name is empty because there is no switch
// 2. Queue name
// 3. Properties when publishing messages - additional settings for delivering messages
// 4. The specific content of the published message shall be an array of bytes,
channel.basicPublish("", "B", null, message.getBytes());Although queue A is bound to channel, queue B actually sends messages. If queue A and queue B are not on the page, they will be created after executing the code.
queueDeclare second parameter:
The following D indicates that it is set to true. Now it is queue persistence. However, after restarting, the message will still be lost.
How to ensure both queue persistence and message persistence?
Persistence of messages.
The third parameter of basicPublish is set to messageproperties PERSISTENT_ TEXT_ PLAIN
The third parameter of queueDeclare:
It is generally false. You want multiple channels to share a queue
The fourth parameter of queueDeclare:
When set to true, the following flag appears. And close the consumer's connection to the queue (close the consumer's program), and the queue will disappear.
3.6 the second model (work queue)
1. Definition:
Work queues, also known as task queues, is a task model. When message processing is time-consuming, the speed of message production may be much faster than that of message consumption. In the long run, more and more messages will accumulate and cannot be processed in time. At this point, you can use the work model: let multiple consumers bind to a queue and consume the messages in the queue together. Once the messages in the queue are consumed, they will disappear, so the task will not be repeated.
2. Role:
● P: Producer: publisher of the task
● C1: the consumer receives and completes the task, assuming that the completion speed is slow
● C2: consumer 2: get the task and complete the task. It is assumed that the completion speed is fast
3. Producers:
public class Provider {
private static final String TASK_QUEUE_NAME = "task_queue";
public static void main(String[] argv) throws Exception {
// Get connection object
Connection connection = RabbitMQUtils.getConnection();
// Gets the channel object in the connection
Channel channel = connection.createChannel();
// Declare queue through channel
channel.queueDeclare(TASK_QUEUE_NAME, true, false, false, null);
// Send 10 messages
for (int i = 0; i < 20; i++) {
channel.basicPublish("", TASK_QUEUE_NAME, null, (i+"How do you do").getBytes());
}
}
}public class Provider {
private static final String TASK_QUEUE_NAME = "task_queue";
public static void main(String[] argv) throws Exception {
// Get connection object
Connection connection = RabbitMQUtils.getConnection();
// Gets the channel object in the connection
Channel channel = connection.createChannel();
// Declare queue through channel
channel.queueDeclare(TASK_QUEUE_NAME, true, false, false, null);
// Send 10 messages
for (int i = 0; i < 20; i++) {
channel.basicPublish("", TASK_QUEUE_NAME, null, (i+"How do you do").getBytes());
}
}
}
4. The consumer creates two: / / the difference is whether to bring the following thread sleep code
public class Customer2 {
private static final String TASK_QUEUE_NAME = "task_queue";
public static void main(String[] args) throws IOException {
// Get connection object
Connection connection = RabbitMQUtils.getConnection();
// Gets the channel object in the connection
Channel channel = connection.createChannel();
// Only one message can be consumed at a time
channel.basicQos(1);
// Channel binding message queue
channel.queueDeclare(TASK_QUEUE_NAME, true, false, false, null);
// Get message parameter 2: automatically confirm the message, and the consumer will automatically confirm the message consumption to rabbitmq (as long as there is a message in the message queue, it will be assigned to the consumer)
channel.basicConsume(TASK_QUEUE_NAME, false, new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("consumer-1: " + new String(body));
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
// Parameter 1: confirm the specific message in the queue parameter 2: whether to enable multiple messages to confirm at the same time
channel.basicAck(envelope.getDeliveryTag(),false);
}
});
}
}5. Summary:
By default, RabbitMQ sends each message to the next consumer in order. On average, each consumer receives the same number of messages. This way of distributing messages is called a loop.
6. Disadvantages:
In this case, when one of them consumes slowly, for example, consumer 3 has finished consuming. But consumers 1 still receive one by one. This will cause the accumulation of messages.
We hope we can handle it more quickly. How?
1. Consumer: to turn off automatic confirmation message
// Get message parameter 2: automatically confirm the message, and the consumer will automatically confirm the message consumption to rabbitmq (as long as there is a message in the message queue, it will be assigned to the consumer, regardless of whether the following code is executed or not)
channel.basicConsume(TASK_QUEUE_NAME, true, new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("consumer-1: " + new String(body));
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
So set the above parameter to false, so
2. You also need to tell the current channel that only one message can be consumed at a time.
// Only one message can be consumed at a time | only one unacknowledged message can be received at a time
channel.basicQos(1);
3. Message confirmation: manually confirm the message
The following occurs because there are few message confirmation codes
// Parameter 1: confirm the specific message in the queue parameter 2: whether to enable multiple messages to confirm at the same time
channel.basicAck(envelope.getDeliveryTag(),false);After the message is confirmed, it is deleted from the queue
3.7. The third model (fanout | broadcast)
fanout is also called broadcast
It is suitable for registration business, such as sending points and SMS authentication.? “？？
For example, when the shopping cart is settled, it is the order system and the inventory system??
1. In broadcast mode, the message sending process is as follows:
● can have multiple consumers
● each consumer has its own queue
● each queue should be bound to Exchange (switch)
● the message sent by the producer can only be sent to the switch, which determines which queue to send, but the producer cannot decide.
● the switch sends messages to all queues bound
● consumers in the queue can get the message. Realize that a message is consumed by multiple consumers
2. Producers
public class Provider {
public static void main(String[] args) throws IOException {
// Get connection object
Connection connection = RabbitMQUtils.getConnection();
// Gets the channel object in the connection
Channel channel = connection.createChannel();
// Specify the channel declaration to switch parameter 1: switch name parameter 2: switch type, and fanout is the broadcast type
channel.exchangeDeclare("logs","fanout");
// send message
channel.basicPublish("logs","",null,"fanout type message".getBytes());
}
}3. Consumers
public class Customer1 {
public static void main(String[] args) throws IOException {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
//Channel bound switch
channel.exchangeDeclare("logs","fanout");
// Create temporary queue
String queue = channel.queueDeclare().getQueue();
// Binding switches and queues
channel.queueBind(queue,"logs","");
// Consumption news
channel.basicConsume(queue,true,new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 1"+new String(body));
}
});
}
}public class Customer2 {
public static void main(String[] args) throws IOException {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
//Channel bound switch
channel.exchangeDeclare("logs","fanout");
// Create temporary queue
String queue = channel.queueDeclare().getQueue();
// Binding switches and queues
channel.queueBind(queue,"logs","");
// Consumption news
channel.basicConsume(queue,true,new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 2"+new String(body));
}
});
}
}Broadcast model: the producer sends a message and consumers 1 and 2 will get it
3.8 Routing model 4
1. Routing subscription model - Direct
In Fanout mode, a message will be consumed by all subscribed queues. However, in some scenarios, we want different messages to be consumed by different queues. In this case, Exchange of Direct type will be used.
Under the Direct model:
● the binding between the queue and the switch cannot be arbitrary, but a RoutingKey should be specified
● when sending a message to Exchange, the sender of the message must also specify the RoutingKey of the message.
● Exchange will no longer hand over the message to each bound queue, but will judge according to the routing key of the message. The message will be received only if the routing key of the queue is completely consistent with the routing key of the message
For example, when an error occurs, it needs to be printed on the console and persisted in the log
Illustration:
● P: the producer sends a message to Exchange. When sending a message, a routing key will be specified.
● x: Exchange (exchange), receive the producer's message, and then submit the message to the queue that exactly matches the routing key
● C1: consumer, whose queue specifies the message whose routing key needs to be error
● C2: consumer, whose queue specifies the messages whose routing key needs to be info, error and warning
2. Producers
public class Provider {
public static void main(String[] args) throws IOException {
// Create connection
Connection connection = RabbitMQUtils.getConnection();
// Create channel
Channel channel = connection.createChannel();
String exchangeName = "logs_direct";
// The channel declaration specifies the switch parameter 1: switch name parameter 2: switch type, and direct is the routing mode
channel.exchangeDeclare(exchangeName,"direct");
// send message
channel.basicPublish(exchangeName,"error",null,("This is direct Model publishing based on route" +
" kye: ["+"info"+"]Messages sent").getBytes());
}
}error, the consumer will receive a message:
info, consumer 2 will receive a message
3. Consumers
public class Customer1 {
public static void main(String[] args) throws IOException {
// 1. Establish connection
Connection connection = RabbitMQUtils.getConnection();
// Establish channel
Channel channel = connection.createChannel();
String exchangeName = "logs_direct";
// Binding switches and channels
channel.exchangeDeclare(exchangeName,"direct");
// Create queue
String queue = channel.queueDeclare().getQueue();
// Bind queue and switch based on routing key
channel.queueBind(queue,exchangeName,"error");
// consumption
channel.basicConsume(queue,true,new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 1:"+ new String(body));
}
});
}
}public class Customer2 {
public static void main(String[] args) throws IOException {
// 1. Establish connection
Connection connection = RabbitMQUtils.getConnection();
// Establish channel
Channel channel = connection.createChannel();
String exchangeName = "logs_direct";
// Binding switches and channels
channel.exchangeDeclare(exchangeName,"direct");
// Create queue
String queue = channel.queueDeclare().getQueue();
// Bind queue and switch based on routing key
channel.queueBind(queue,exchangeName,"info");
channel.queueBind(queue,exchangeName,"error");
channel.queueBind(queue,exchangeName,"warning");
// consumption
channel.basicConsume(queue,true,new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 2:"+ new String(body));
}
});
}
}3.9. Fifth model (Topics) dynamic routing
Because the fourth kind of consumer is so inflexible: write one line of code
1. Definition:
Compared with Direct, Topic Exchange can route messages to different queues according to the Routingkey. However, Topic type Exchange allows the queue to use wildcards when Binding routing keys. This model Routingkey is generally composed of one or more words, with "." between multiple words Split, for example: item insert
# wildcard * (star) can substitute for exactly one word. Match no more, no less, exactly one word # (hash) can substitute for zero or more words. Match 0, 1 or more words #For example: audit.# Match audit irs . Corporate or audit IRS et al audit.* Only match audit.irs
2. The producer is currently user save
public class Provider {
public static void main(String[] args) throws IOException {
// Establish connection factory
Connection connection = RabbitMQUtils.getConnection();
// Establish channel
Channel channel = connection.createChannel();
String exchangeName = "topics";
// Switch
channel.exchangeDeclare(exchangeName,"topic");
// Send a message
String routekey = "user.save";
channel.basicPublish(exchangeName,routekey,null,("This is topic Dynamic routing model publishing based on route" +
" key :["+routekey+"]").getBytes());
}
}3. Consumers
public class Customer1 {
public static void main(String[] args) throws IOException {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
//Channel bound switch
channel.exchangeDeclare("topics","topic");
// Create temporary queue
String queue = channel.queueDeclare().getQueue();
// Binding switches and queues
channel.queueBind(queue,"topics","user.*");
// Consumption news
channel.basicConsume(queue,true,new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 1"+new String(body));
}
});
}
}public class Customer2 {
public static void main(String[] args) throws IOException {
Connection connection = RabbitMQUtils.getConnection();
Channel channel = connection.createChannel();
//Channel bound switch
channel.exchangeDeclare("topics","topic");
// Create temporary queue
String queue = channel.queueDeclare().getQueue();
// Binding switches and queues
channel.queueBind(queue,"topics","user.#");
// Consumption news
channel.basicConsume(queue,true,new DefaultConsumer(channel){
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("Consumer 2"+new String(body));
}
});
}
}