In the project, there is an intermittent condition in the ActiveMQ consumer log, and there is no traceId output

In this way, there is no link to track the problem

Cause: the link is not passed from the main thread to the sub thread

Solution:

@TraceCrossThread

@The annotations in the toolkit provided by TraceCrossThread for skywalking can be considered intrusive. The use method is to add the annotation on the thread class. When the class is loaded, its construction method will be enhanced by the agent agent

@TraceCrossThread
public class MyRunnable implements Runnable{
  @Override
  public void run() {
      System.out.println(TraceContext.traceId());
      doNothing();
  }
}

However, it is not appropriate to change the ActiveMQ source code in this way

apm-jdk-threading-plugin

APM JDK threading plugin is an official plug-in, which is really non intrusive and easy to use. This plug-in is located in ${skywalking_dir} / agent / bootstrap plugins directory. All we need to do is copy it to ${skywalking_dir}/agent/plugins directory.

In addition, you need to modify the agent configuration ${skywalking_dir} / agent / config / agent Config, tell the agent to enhance the thread pools under those packages. The configuration is as follows:

plugin.jdkthreading.threading_class_prefixes=${SW_PLUGIN_JDKTHREADING_THREADING_CLASS_PREFIXES:}

The system environment variable can be configured by configuration override:

SW_PLUGIN_JDKTHREADING_THREADING_CLASS_PREFIXES=org.springframework.jms.listener.DefaultMessageListenerContainer

reference resources: Configure override

ActiveMQ consumption message log hold tid output

Or is it to borrow that there is no tid in the log output of ActiveMQ consumer? After understanding the source code of skywalking plugin, it is necessary to modify activemq-5 Transform the official plug-in of x-plugin

1. Add the ActiveMessageListenerInstrumentation class to enable probes

#skywalking-plugin.def
activemq-5.x=org.apache.skywalking.apm.plugin.activemq.define.ActiveMessageListenerInstrumentation

public class ActiveMessageListenerInstrumentation extends ClassInstanceMethodsEnhancePluginDefine {
    public static final String INTERCEPTOR_CLASS = "org.apache.skywalking.apm.plugin.activemq.ActiveMessageListenerInterceptor";
    public static final String ENHANCE_CLASS_CONSUMER = "org.springframework.jms.listener.DefaultMessageListenerContainer";
    public static final String ENHANCE_METHOD_DISPATCH = "doExecuteListener";

    @Override
    public ConstructorInterceptPoint[] getConstructorsInterceptPoints() {
        return new ConstructorInterceptPoint[0];
    }

    @Override
    public InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() {
        return new InstanceMethodsInterceptPoint[] {
            new InstanceMethodsInterceptPoint() {
                @Override
                public ElementMatcher<MethodDescription> getMethodsMatcher() {
                    return named(ENHANCE_METHOD_DISPATCH);
                }

                @Override
                public String getMethodsInterceptor() {
                    return INTERCEPTOR_CLASS;
                }

                @Override
                public boolean isOverrideArgs() {
                    return false;
                }
            }
        };
    }

    @Override
    protected ClassMatch enhanceClass() {
        return MultiClassNameMatch.byMultiClassMatch(ENHANCE_CLASS_CONSUMER);
    }
}

1. Key codes:

ENHANCE_CLASS_CONSUMER = "org.springframework.jms.listener.DefaultMessageListenerContainer";
ENHANCE_METHOD_DISPATCH = "doExecuteListener";

After understanding the source code of Spring jms, it is learned that DefaultMessageListenerContainer is a reusable thread for asynchronous message monitoring, and the real consumption message is the last call to its parent AbstractMessageListenerContainer#doExecuteListener method.

1. Specific method interception logic

public class ActiveMessageListenerInterceptor implements InstanceMethodsAroundInterceptor {
​
  public static final String OPERATE_NAME_PREFIX = "ActiveMQ/";
  public static final String CONSUMER_OPERATE_NAME_SUFFIX = "/OnMessage";
​
  @Override
  public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class<?>[] argumentsTypes,
      MethodInterceptResult result) throws Throwable {
      ContextCarrier contextCarrier = new ContextCarrier();
      Message message = (Message) allArguments[1];
      CarrierItem next = contextCarrier.items();
      while (next.hasNext()) {
          next = next.next();
          Object propertyValue = message.getStringProperty(next.getHeadKey());
          if (propertyValue != null) {
              next.setHeadValue(propertyValue.toString());
          }
      }
      AbstractSpan localSpan = ContextManager.createEntrySpan(OPERATE_NAME_PREFIX + message.getJMSDestination() + CONSUMER_OPERATE_NAME_SUFFIX, contextCarrier);
      localSpan.setComponent(ComponentsDefine.ACTIVEMQ_CONSUMER);
      SpanLayer.asMQ(localSpan);
      ContextManager.extract(contextCarrier);
  }
 
···Omitted below

The interception logic mainly parses the attributes in the Message, because skywalking will add relevant attributes to the mq header.

Create an EntrySpan through the ContestManager#createEntrySpan method, and finally use ContextManager#extract to establish a distributed call Association;

• Reference 1: org apache. skywalking. apm. plugin. activemq. Activemqconsumerinterceptor source code

• Reference 2: Plug in Development Guide

1. Finally, replace the plug-in after the package to: / APP / skywalking / agent / plugins / apm-activemq-5 x-plugin-xxx. jar