An InterruptException occurred when the RocketMQConsumer service was shut down
Background summary
The main problem is the version upgrade
-
Old version core rocketmq dependency
<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>spring-boot-starter-rocketmq</artifactId> <version>${vesion}</version> </dependency> <dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-client</artifactId> <version>4.3.2</version> </dependency> -
New version of core rocketmq dependency
<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-client</artifactId> <version>4.9.2</version> </dependency> <dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-spring-boot-starter</artifactId> <version>2.2.1</version> </dependency>
java.lang.InterruptedException
Simply enumerate an InterruptedException
java.sql.SQLException: interrupt at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1430) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1272) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:5007) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.filter.FilterAdapter.dataSource_getConnection(FilterAdapter.java:2745) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:5003) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.java:680) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:5003) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1250) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1242) ~[druid-1.1.12.jar!/:1.1.12] at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:89) ~[druid-1.1.12.jar!/:1.1.12] // Omit some stack information at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.handleMessage(DefaultRocketMQListenerContainer.java:399) [rocketmq-spring-boot-2.2.1.jar!/:2.2.1] at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.access$100(DefaultRocketMQListenerContainer.java:71) [rocketmq-spring-boot-2.2.1.jar!/:2.2.1] at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer$DefaultMessageListenerConcurrently.consumeMessage(DefaultRocketMQListenerContainer.java:359) [rocketmq-spring-boot-2.2.1.jar!/:2.2.1] at cn.techwolf.trace.rocketmq.spring.TracingMessageListenerConcurrently.consumeMessage(TracingMessageListenerConcurrently.java:37) [instrument-rocketmq-spring-1.101.jar!/:1.101] at org.apache.rocketmq.client.impl.consumer.ConsumeMessageConcurrentlyService$ConsumeRequest.run(ConsumeMessageConcurrentlyService.java:392) [rocketmq-client-4.9.2.jar!/:4.9.2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_202] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_202] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_202] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_202] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_202] Caused by: java.lang.InterruptedException
be careful
The following analysis is only a personal point of view and is not necessarily correct. If there is a bar essence, please do not continue to watch it. Welcome to leave a message for discussion
Tip: some classes involved in this article are A NullPointerException problem was recorded when the RocketMQ service was started This article does not do some detailed explanation
spring shutdown & rocketmq shutdown timing (interleaving is not important)
-
spring
-
rocketmq inherits SmartCycle and calls back its stop method when the container is closed
rocketmq shutdown analysis
First, confirm the implementation of our RocketMQConsumer. Defaultrocketmqllistenercontainer:
// Class full path org apache. rocketmq. spring. support. DefaultRocketMQListenerContainer
public class DefaultRocketMQListenerContainer implements InitializingBean, RocketMQListenerContainer, SmartLifecycle, ApplicationContextAware {
private DefaultMQPushConsumer consumer;
@Override
public void destroy() {// Called when the DisposableBean callback destroys the bean
this.setRunning(false);
if (Objects.nonNull(consumer)) {
consumer.shutdown();
}
log.info("container destroyed, {}", this.toString());
}
@Override
public void stop() { // SmartLifecycle callback callback before closing container
if (this.isRunning()) {
if (Objects.nonNull(consumer)) {
consumer.shutdown();
}
setRunning(false);
}
}
}
It can be seen that the defaultrocketmqllistenercontainer is called first if it is closed Stop method. The next step is to look at consumer The shutdown () method:
// Class full path org apache. rocketmq. client. consumer. DefaultMQPushConsumer
public class DefaultMQPushConsumer extends ClientConfig implements MQPushConsumer {
protected final transient DefaultMQPushConsumerImpl defaultMQPushConsumerImpl;
/**
* Maximum time to await message consuming when shutdown consumer, 0 indicates no await.
*/
private long awaitTerminationMillisWhenShutdown = 0;
@Override
public void shutdown() {
this.defaultMQPushConsumerImpl.shutdown(awaitTerminationMillisWhenShutdown);
if (null != traceDispatcher) {
traceDispatcher.shutdown();
}
}
}
// Class full path org apache. rocketmq. client. impl. consumer. DefaultMQPushConsumerImpl
public class DefaultMQPushConsumerImpl implements MQConsumerInner {
private ConsumeMessageService consumeMessageService;
public synchronized void shutdown(long awaitTerminateMillis) {
switch (this.serviceState) {
case CREATE_JUST:
break;
case RUNNING:
this.consumeMessageService.shutdown(awaitTerminateMillis);
this.persistConsumerOffset();
this.mQClientFactory.unregisterConsumer(this.defaultMQPushConsumer.getConsumerGroup());
this.mQClientFactory.shutdown();
log.info("the consumer [{}] shutdown OK", this.defaultMQPushConsumer.getConsumerGroup());
this.rebalanceImpl.destroy();
this.serviceState = ServiceState.SHUTDOWN_ALREADY;
break;
case SHUTDOWN_ALREADY:
break;
default:
break;
}
}
}
// Consummessageservice has two implementation classes: consummessageconcurrentlyservice and consummessageorderlyservice
// The specific implementation class of consumeMessageService we use is consumemessageconcurrent service. Specific analysis of specific problems
// Class full path org apache. rocketmq. client. impl. consumer. ConsumeMessageConcurrentlyService
public class ConsumeMessageConcurrentlyService implements ConsumeMessageService {
private final ThreadPoolExecutor consumeExecutor;
public void shutdown(long awaitTerminateMillis) {
this.scheduledExecutorService.shutdown();
ThreadUtils.shutdownGracefully(this.consumeExecutor, awaitTerminateMillis, TimeUnit.MILLISECONDS);
this.cleanExpireMsgExecutors.shutdown();
}
}
// Class full path org apache. rocketmq. common. utils. ThreadUtils
public final class ThreadUtils {
public static void shutdownGracefully(ExecutorService executor, long timeout, TimeUnit timeUnit) {
// Disable new tasks from being submitted.
executor.shutdown();
try {
// Wait a while for existing tasks to terminate.
if (!executor.awaitTermination(timeout, timeUnit)) { // Pay attention here
executor.shutdownNow();
// Wait a while for tasks to respond to being cancelled.
if (!executor.awaitTermination(timeout, timeUnit)) {
log.warn(String.format("%s didn't terminate!", executor));
}
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted.
executor.shutdownNow();
// Preserve interrupt status.
Thread.currentThread().interrupt();
}
}
}
From the above, we can clearly see that consumemessageconcurrent service During shutdown, the default value of awaitterminatemilis is 0, which is executed to threadutils When shutdowngracefully, shutdownnow will be called directly without waiting
What's the difference between shutdown and shutdown now? Just search Baidu
- Shutdown = > gently shut down and wait for all tasks added to the thread pool to finish executing
- Shutdown now = > close immediately, stop the executing task, and return the unexecuted task in the queue
Compare with the previous code rocketmq client 4.3.2
public class ConsumeMessageConcurrentlyService implements ConsumeMessageService {
private final ThreadPoolExecutor consumeExecutor;
public void shutdown(long awaitTerminateMillis) {
this.scheduledExecutorService.shutdown();
this.consumeExecutor.shutdown(); // here
this.cleanExpireMsgExecutors.shutdown();
}
}
Therefore, I think it is because the interrupt InterruptException exception occurs when the service is shut down due to the direct call to shutdownnow
Solution
From the above analysis, we can know that it is caused by calling shutdown now directly. We should be able to adjust the awaitterminatemilis parameter, that is, defaultmqpushconsumer Awaitterminationmilliswhenshutdown parameter
At present, I don't seem to see any way to support the global configuration of this parameter (official way). Here are two ideas
RocketMQPushConsumerLifecycleListener or RocketMQPushConsumerLifecycleListener
// org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer
public class DefaultRocketMQListenerContainer implements InitializingBean, RocketMQListenerContainer, SmartLifecycle, ApplicationContextAware {
@Override
public void afterPropertiesSet() throws Exception {
initRocketMQPushConsumer();
this.messageType = getMessageType();
this.methodParameter = getMethodParameter();
log.debug("RocketMQ messageType: {}", messageType);
}
private void initRocketMQPushConsumer() throws MQClientException {
// Omit some codes
if (Objects.nonNull(rpcHook)) {
consumer = new DefaultMQPushConsumer(consumerGroup, rpcHook, new AllocateMessageQueueAveragely(),
enableMsgTrace, this.applicationContext.getEnvironment().
resolveRequiredPlaceholders(this.rocketMQMessageListener.customizedTraceTopic()));
consumer.setVipChannelEnabled(false);
} else {
log.debug("Access-key or secret-key not configure in " + this + ".");
consumer = new DefaultMQPushConsumer(consumerGroup, enableMsgTrace,
this.applicationContext.getEnvironment().
resolveRequiredPlaceholders(this.rocketMQMessageListener.customizedTraceTopic()));
}
// Omit some consumer parameter configuration codes
// You can see that he will call back the prepareStart method
if (rocketMQListener instanceof RocketMQPushConsumerLifecycleListener) {
((RocketMQPushConsumerLifecycleListener) rocketMQListener).prepareStart(consumer);
} else if (rocketMQReplyListener instanceof RocketMQPushConsumerLifecycleListener) {
((RocketMQPushConsumerLifecycleListener) rocketMQReplyListener).prepareStart(consumer);
}
}
}
You can see that after the defaultrocketmqllistenercontainer is initialized, the prepareStart methods of rocketmqppushconsumerlifcyclelistener and rocketmqppushconsumerlifcyclelistener will be called back
That's easy
@Service
@RocketMQMessageListener(nameServer = "${spring.rocketmq.nameServer}",
topic = "${topic}",
consumerGroup = "${group}")
public class TestConsumer implements RocketMQListener<String>, RocketMQPushConsumerLifecycleListener {
@Override
public void onMessage(String msg) {
// do something
}
@Override
public void prepareStart(DefaultMQPushConsumer consumer) {
consumer.setAwaitTerminationMillisWhenShutdown(1000); // set up
}
}
It can be abstracted into a general class, and the consumer can inherit this class
DefaultRocketMQListenerContainer.getConsumer
Defaultrocketmqllistenercontainer will be registered as a bean, and the specific implementation is at org apache. rocketmq. spring. autoconfigure. In ListenerContainerConfiguration, there is no analysis here. We can try to get all the DefaultRocketMQListenerContainer and call the getConsumer method. The sample code can trigger the code in detail.
public void set() {
List<DefaultRocketMQListenerContainer> containers = getAllBeans();
for (DefaultRocketMQListenerContainer c : containers) {
c.getConsumer().setAwaitTerminationMillisWhenShutdown(1000);
}
}
Finally, the above analysis is only for me and is not necessarily correct. In the first way, there is no InterruptException. I don't know whether it is accidental;
Welcome to leave a message for discussion