1. Introduction
1.1 INTRODUCTION
SandboxRepeater is a set of traffic playback tools. The open source ones should not be updated basically. The projects charged by Alibaba are coming out and are still in the public beta stage. At present, the open source traffic playback frameworks include Didi's Rdebub and Alibaba's SandboxRepeater. However, other frameworks like Didi's are PHP frameworks. Unless they are perfectly adapted, the SpringBoot project can only be modified based on SandboxRepeater if it is used.
1.2 architecture
- Plug in
- Console interface console
1.3 how to start Sandbox Repeater client
2. Source code
2.1 installing Sandbox
- Remote installation: remotely download the install repeater file through the command to install the client. This is available in the bin directory of Alibaba open source project and can be run directly.
curl -s http://sandbox-ecological.oss-cn-hangzhou.aliyuncs.com/install-repeater.sh | sh
After downloading, it will be installed into the ${HOME}/sandbox directory ("/ user / apple / sandbox") according to the command in install repeater. The directory structure is as follows
2.2 start Sandbox
Sandbox is mainly based on the principle of java agent. The startup methods are divided into attach and agent. One is to start without the command of the target application, and the other is to start as the JVM parameter of the target application. The characteristics of the two methods are different. Sandbox startup requires one port.
2.2.1 attach mode
In attach mode, the two parameters of recording application name and recording environment will be defaulted to unknown.
# Start command
~/sandbox/bin/sandbox.sh -p ${Recorded application process number} -P ${repeater Boot port}
# close command
~/sandbox/bin/sandbox.sh -S ${Recorded application process number}(1) Why is the default recording application name and recording environment device unknown
The system parameters of the target application are obtained in the Repeater. If the target application is started in the attach mode, the parameters required by the Repeater will not be configured in advance. Traffic playback mainly simulates the real environment data for positioning. If you really want to use it, the first method is more friendly. It is mainly that there is no immersion in the target application, and you can turn on or turn off the recording at any time, while the second method is consistent with the life cycle of the target application.
// com.alibaba.jvm.sandbox.repeater.plugin.core.model.ApplicationModel#ApplicationModel
private ApplicationModel() {
// for example, you can define it your self
this.appName = getSystemPropertyOrDefault("app.name", "unknown");
this.environment = getSystemPropertyOrDefault("app.env", "unknown");
try {
this.host = InetAddress.getLocalHost().getHostAddress();
} catch (UnknownHostException e) {
// default value for disaster
this.host = "127.0.0.1";
}
}2.2.2 agent mode
JVM parameters are configured as follows:
-javaagent:${HOME}/Desktop/sandboxb/lib/sandbox-agent.jar=server.port=8820;server.ip=0.0.0.0
-Dapp.name=unknown
-Dapp.env=unknown2.3 Repeater
Sandbox will select server according to the parameter Port and server IP to start. The command mode can be configured or sandbox can be from cfg Configuration in properties (I didn't see which is effective and how it works). Sandbox is only used as an entry. It will configure sandbox according to the configuration Properties read user_module information, load the jar package information under this folder. The Repeater will load the Repeater under the changed directory Properties information (this file contains some configuration information such as heartbeat reporting, configuration pull, playback result delivery, etc.)
#
# this properties file define the sandbox's config
# @author luanjia@taobao.com
# @date 2016-10-24
#
# define the sandbox's ${SYSTEM_MODULE} dir
## system_module=../module
# define the sandbox's ${USER_MODULE} dir, multi values, use ',' split
## user_module=~/.sandbox-module;~/.sandbox-module-1;~/.sandbox-module-2;~/.sandbox-module-n;
user_module=~/.sandbox-module;
#user_module=/home/staragent/plugins/jvm-sandbox-module/sandbox-module;/home/staragent/plugins/monkeyking;
# define the sandbox's ${PROVIDER_LIB} dir
## provider=../provider
# define the network interface
## server.ip=0.0.0.0
# define the network port
## server.port=4769
# switch the sandbox can enhance system class
unsafe.enable=true
2.4 matching method and interception conditions
There are two methods of method matching, one is wildcard (a.b.c. *) and the other is regular matching. The default matching method is wildcard.
{
"useTtl" : true,
"degrade" : false,
"exceptionThreshold" : 1000,
"sampleRate" : 0,
"pluginsPath" : null,
"httpEntrancePatterns" : [ "^/regress/.*$" ],
"javaEntranceBehaviors" : [ {
"classPattern" : "com.alibaba.repeater.console.service.impl.*",
"methodPatterns" : [ "*" ],
"includeSubClasses" : false
} ],
"javaSubInvokeBehaviors" : [ {
"classPattern" : "com.alibaba.repeater.console.service.impl.RegressServiceImpl",
"methodPatterns" : [ "getRegressInner", "findPartner", "slogan" ],
"includeSubClasses" : false
} ],
"pluginIdentities" : [ "http", "java-entrance", "java-subInvoke" ],
"repeatIdentities" : [ "java", "http" ]
}(1) Wildcard comparison
// com.alibaba.jvm.sandbox.api.util.GaStringUtils#matching(java.lang.String, java.lang.String)
public static boolean matching(final String string, final String wildcard) {
return null != wildcard
&& null != string
&& matching(string, wildcard, 0, 0);
}
// com.alibaba.jvm.sandbox.api.util.GaStringUtils#matching(java.lang.String, java.lang.String, int, int)
private static boolean matching(String string, String wildcard, int stringStartNdx, int patternStartNdx) {
int pNdx = patternStartNdx;
int sNdx = stringStartNdx;
int pLen = wildcard.length();
if (pLen == 1) {
if (wildcard.charAt(0) == '*') { // speed-up
return true;
}
}
int sLen = string.length();
boolean nextIsNotWildcard = false;
while (true) {
// check if end of string and/or pattern occurred
if ((sNdx >= sLen)) { // end of string still may have pending '*' callback pattern
while ((pNdx < pLen) && (wildcard.charAt(pNdx) == '*')) {
pNdx++;
}
return pNdx >= pLen;
}
if (pNdx >= pLen) { // end of pattern, but not end of the string
return false;
}
char p = wildcard.charAt(pNdx); // pattern char
// perform logic
if (!nextIsNotWildcard) {
if (p == '\\') {
pNdx++;
nextIsNotWildcard = true;
continue;
}
if (p == '?') {
sNdx++;
pNdx++;
continue;
}
if (p == '*') {
char pnext = 0; // next pattern char
if (pNdx + 1 < pLen) {
pnext = wildcard.charAt(pNdx + 1);
}
if (pnext == '*') { // double '*' have the same effect as one '*'
pNdx++;
continue;
}
int i;
pNdx++;
// find recursively if there is any substring from the end of the
// line that matches the rest of the pattern !!!
for (i = string.length(); i >= sNdx; i--) {
if (matching(string, wildcard, i, pNdx)) {
return true;
}
}
return false;
}
} else {
nextIsNotWildcard = false;
}
// check if pattern char and string char are equals
if (p != string.charAt(sNdx)) {
return false;
}
// everything matches for now, continue
sNdx++;
pNdx++;
}
}
2.5 event monitoring
The enhancements of any method can be divided into three types: before method, after method and when abnormal. sandbox abstracts the event behavior and provides event listening behavior. Repeater is implemented in this way. The plug-in can override event listening for custom processing.
// com.alibaba.jvm.sandbox.repeater.plugin.core.impl.api.DefaultEventListener#onEvent
public void onEvent(Event event) throws Throwable {
try {
/*
* event Filtration; For a single listener, only top events are processed
*/
if (!isTopEvent(event)) {
// ...
return;
}
/*
* Initialize Tracer
*/
initContext(event);
/*
* Perform basic filtering
*/
if (!access(event)) {
// ...
return;
}
/*
* Execute the sampling calculation (only the entry plug-in is responsible for calculating the sampling, and the sub calling plug-in does not calculate). The traceId remains unchanged, while the sampling calculation is calculated according to the traceId. Therefore, part of the event sampling passes and part fails will not occur in a complete call
*/
if (!sample(event)) {
// ...
return;
}
/*
* processor filter
*/
if (processor != null && processor.ignoreEvent((InvokeEvent) event)) {
// ...
return;
}
/*
* Distribution event processing (for an around event, the input / return values that can be collected can be used directly; they need to be obtained from multiple before practices)
*/
switch (event.type) {
case BEFORE:
// Record call information
doBefore((BeforeEvent) event);
break;
case RETURN:
// Record the call information. After all closed loops, the HTTP call recording interface saves the relevant call information. The address information is from the repeater Read properties (thread pool is processed asynchronously, and attention should be paid to the serialization method when saving)
doReturn((ReturnEvent) event);
break;
case THROWS:
// Same as return
doThrow((ThrowsEvent) event);
break;
default:
break;
}
} catch (ProcessControlException pe) {
/*
* sandbox Process intervention
*/
// process control will interrupt the event and there will be no return/throw event, so you need to clear the offset
eventOffset.remove();
throw pe;
} catch (Throwable throwable) {
// uncaught exception
log.error("[Error-0000]-uncaught exception occurred when dispatch event,type={},event={}", invokeType, event, throwable);
ApplicationModel.instance().exceptionOverflow(throwable);
} finally {
/*
* Portal plug-in & & completion event
*/
clearContext(event);
}
}2.6 playback
In case of playback traffic, initiate mock and build mock information.
// com.alibaba.jvm.sandbox.repeater.plugin.core.impl.api.DefaultEventListener#doBefore
protected void doBefore(BeforeEvent event) throws ProcessControlException {
// Playback traffic; If it is an entrance, give up; The child call is mock
if (RepeatCache.isRepeatFlow(Tracer.getTraceId())) {
processor.doMock(event, entrance, invokeType);
return;
}
// ...
}3. Practice
4. FAQ
4.1 Redis plug-in can only intercept Jedis connection mode, but cannot intercept if using Lettuce mode
4.2 SandboxRepeater can record JAVA method input and return results, but only if these recorded objects have the qualification to be serialized and deserialized, such as Hession and Jason. At present, if the parameter contains HttpServerletRequest, recording cannot be carried out (the recorded input information will be missing),
Then, whether HttpServerletRequest has means to store for playback.
4.3 there is a sampling rate configuration for recording, according to traceid% 10000 < sampling base.
4.4 ， sometimes it is only used as a small tool and does not want to make a big deal. At present, it is still troublesome to use this set. It seems that some functions of Arthas overlap. You can see it next time (TBD).
4.5 why is it only for the top event? How to distinguish the methods of sub calls
4.6 TTL
4.7 whether the sandbox sends messages once for all subscription events (the plug-in can redefine event listening)