Focus on official account dry cargo WeChat public: K brother crawler, keep sharing crawler advance, JS/ Android reverse technology dry goods!
statement
All contents in this article are for learning and communication only. The packet capturing content, sensitive website and data interface have been desensitized. It is strictly prohibited to use them for commercial and illegal purposes, otherwise all the consequences have nothing to do with the author. If there is infringement, please contact me and delete them immediately!
Reverse target
- Objective: unlimited debugger of an air quality monitoring platform and dynamic encryption and decryption of request data and return data
- Home page: aHR0cHM6Ly93d3cuYXFpc3R1ZHkuY24v
- Interface: aHR0cHM6Ly93d3cuYXFpc3R1ZHkuY24vYXBpbmV3L2FxaXN0dWR5YXBpLnBocA==
Write in front
This site is updated frequently. Many bloggers have written analysis articles on the site before brother K. recently, some readers asked for data encryption and decryption of returned data, and found that the encryption and decryption JS has become dynamic. The solutions mentioned in previous articles are not very good, but on the whole, it is not very difficult, but it is a little troublesome to deal with, There are some small details to pay attention to.
It can be seen from the "about system" of the website that this website seems to be maintained by individual developers, which was first established in 2013. In the friendship sponsorship list, it can be seen that most of them are university professionals and researchers related to environment, surveying and mapping and public health. It can be guessed that these data are very helpful for their research, Coupled with the frequent anti climbing updates, it can be seen that the webmaster suffers from crawlers, and brother K doesn't want to add a burden to the webmaster. After all, we should support this kind of site and let him maintain it for a long time. Therefore, brother K only analyzes logic and a small part of the code in this issue and doesn't put the complete code. If there are relevant professionals who really need to grab data for research, I can contact me in the official account.
Bypass infinite debugger
Right click F12 and you will be prompted that the right button is disabled. It doesn't matter. Use the shortcut Ctrl+Shift+i or the upper right corner of the browser. More tools and developer tools can still be opened.
Method 1
When you open the console, you will enter the first infinite debugger. You can see a try-catch statement as you go up to a stack. If you break the point, you will find that he will go catch all the time, calling the setTimeout() method. This method is used to call functions or calculate expressions after the specified milliseconds, and notice that the debugger is passed to the construction method constructor. So here we have two methods to get past debugger, Hook, constructor or setTimeout.
// Choose one of the two hooks
// Hook construction method
Function.prototype.constructor_ = Function.prototype.constructor;
Function.prototype.constructor = function (a) {
if(a == "debugger") {
return function (){};
}
return Function.prototype.constructor_(a);
};
// Hook setTimeout
var setTimeout_ = setTimeout
var setTimeout = function (func, time){
if (func == txsdefwsw){
return function () {};
}
return setTimeout_(func, time)
}
Then we came to the second infinite debugger, which is also similar to the stack. We found a setInterval timer and constructor. Similarly, we can Hook up and drop constructor or setInterval. Note: the timer also detects the window height and width. Even if you pass the constructor or setInterval, you can't take out the developer tool alone. It will continue to output "illegal debugging detected".
// Hook setInterval
var setInterval_ = setInterval
setInterval = function (func, time){
if (time == 2000) {
return function () {};
}
return setInterval_(func, time)
}
We have observed that these two infinite debugger s can pass through the Hook constructor, so we can directly inject the code of the Hook constructor into Fiddler:
Method 2
When we encounter the second infinite debugger, we can also directly follow the stack to a city_ realtime. There are two Eval statements in the PHP page. When you execute the first Eval statement, you will find that it is the debugger code we saw in the VM virtual machine. Therefore, in theory, you can directly replace this page. If you remove the eval statement, there will be no infinite debugger, but brother K told you first that it can't work now, Because there is a JS loaded in it, this JS will be used in later encryption and decryption, but this JS is dynamic and will change every 10 minutes. We need to obtain the dynamic JS through this page later, so it cannot be replaced! Here is just a mention of this idea!
Method 3
Of course, there is also the simplest method. Right click and select Never pause here. You also need to take out the developer tool window separately, otherwise "illegal debugging detected" will be output all the time.
Packet capture analysis
On the real-time monitoring page, click to query a city, and you can see that the requested Form Data and the returned data are encrypted, as shown in the figure below:
Encryption entry
Since it is XHR, we can easily find the encrypted location by directly following the stack:
You can see the passed data key value pair: {hXM8NDFHN: p7crXYR}. The key is written dead in this JS. The value is obtained through a method pu14vhqrofrulds(). This method needs to pass two parameters. The first is the fixed value GETDATA and the second is the city name. Let's follow up to see what this method is:
For some appId, timestamp, city and other parameters, MD5 and base64 operations are performed, and the returned param is the value we want. It doesn't seem difficult. Let's find out how the returned encrypted data is decrypted. We notice that the ajax request has a success keyword. Even if we don't understand JS logic, we can guess that it should be the processing operation after the request is successful, as shown in the figure below: the incoming dzJMI is the returned encrypted data. After passing through the db0HpCYIy97HkHS7RkhUn() method, Decryption succeeded:
Follow up the db0HpCYIy97HkHS7RkhUn() method. You can see that AES+DES+BASE64 decrypts. The incoming key and offset iv are defined in the header:
Dynamic JS
After the above analysis, we have completed the encryption and decryption logic. However, if you debug more, you will find that the JS for encryption and decryption changes dynamically, and the defined key and offset iv will change every other period of time. If you break the point in this code for a long time, and suddenly find that the breakpoint fails and cannot be broken, that is, the JS has changed, The current code has expired.
We randomly collect two different JS (hint: JS will change every 10 minutes, which will be analyzed in detail later). Using PyCharm's file comparison function (select View - Compare With in turn), we can summarize the following changes (the change of variable name does not count):
- The values of the first 8 parameters: two AES keys and iv, two des key s and iv;
- When generating an encrypted param, the appId changes. The final encryption is divided into AES, DES and no encryption (this is the most easily ignored place. It is not noticed here. The request may prompt that the appId is invalid):
- When the request is finally sent, the data key value pair, in which the key is also changed:
We have found the changes. How can we get this JS? Because this JS is in the VM, we have to find its source and where it comes from. We can see a special JS, similar to encrypt, when we grab the package_ xxxxxx. JS, it's not easy to see the name. It returns the code of an eval package:
We are already familiar with eval. Remove Eval directly and let it execute. You can see that it is the JS we need:
Here is a small detail. If you use the console, you will find that it is printing img tags all the time, which affects our input. You can directly follow in here and stop it from running temporarily at the next breakpoint. There is no need to do other operations to waste time:
You think it's almost done here? Wrong, the same encrypt_xxxxxx.js also has a mystery:
- encrypt_ xxxxxx. The name of JS is dynamic, and the subsequent v value is a second timestamp. It will change every 600 seconds, that is, ten minutes. This JS can be displayed in city_realtime.php page found. Remember we said that bypassing infinite debugger can't replace this page? We want to get dynamic JS through this page, so it can't be replaced!
- encrypt_ xxxxxx. Not all JS returned by JS can get plaintext code by executing Eval once. It is a combination of Eval and base64. The first time is Eval, but it is uncertain later. It may directly produce results. It may need base64, it may need base64 twice, and it may need Eval after base64 twice. In short, except that the first time is Eval, Whether base64 and eval are needed later, as well as the number and order of needs, are uncertain! For example:
Someone here may ask, how can you see that it's base64? Simply enter dswejwehxt in the console of the website page, and click to see this function, which is base64:
So for encrypt_xxxxxx.js content is uncertain, we can write a method to get encrypt_xxxxxx.js, execute eval if you need to execute eval, and execute base64 if you need to execute base64 until there are no eval and base64. You can use the string eval(function) and dswejwehxt (to judge whether eval and base64 are needed (of course, there are other methods, such as the number of ()), and the example code is as follows:
def get_decrypted_js(encrypted_js_url):
"""
:param encrypted_js_url: encrypt_xxxxxx.js Address of
:return: Decrypted JS
"""
decrypted_js = requests.get(url=encrypted_js_url, headers=headers).text
flag = True
while flag:
if "eval(function" in decrypted_js:
# eval is required
print("Need to execute eval!")
replace_js = decrypted_js.replace("eval(function", "(function")
decrypted_js = execjs.eval(replace_js)
elif "dswejwehxt(" in decrypted_js:
# base64 decoding required
base64_num = decrypted_js.count("dswejwehxt(")
print("need %s second base64 decode!" % base64_num)
decrypted_js = re.findall(r"\('(.*?)'\)", decrypted_js)[0]
num = 0
while base64_num > num:
decrypted_js = base64.b64decode(decrypted_js).decode()
num += 1
else:
# Get plaintext
flag = False
# print(decrypted_js)
return decrypted_js
Local overwrite
Through the above functions, we get the dynamic JS. Can we directly execute the JS we get back? Of course not. You can execute it locally and find CryptoJS, Base64 and Hex in it_ MD5 needs to be supplemented, so here we have two methods:
- After getting the decrypted dynamic JS, the dynamic JS and Base64 and hex written by ourselves_ MD5 and other methods form a new JS code, execute the new JS code and get the parameters. It should also be noted here that because other method names are dynamic, you have to find a way to match the correct method name to call, so this method is still a little troublesome;
- We write a JS locally. After we get the decrypted dynamic JS, we match the key, iv, appId, data key name and param whether they need AES or DES encryption. Then we send them to the JS we write and call our own methods to get the encryption results.
Although both methods are troublesome, brother K can't think of a better solution for the time being. Friends with better ideas can leave a message.
Take the second method as an example, our local JS example (main.js):
var CryptoJS = require("crypto-js");
var BASE64 = {
encrypt: function (text) {
return CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(text))
},
decrypt: function (text) {
return CryptoJS.enc.Base64.parse(text).toString(CryptoJS.enc.Utf8)
}
};
var DES = {
encrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(0, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(24, 8);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.DES.encrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString();
},
decrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(0, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(24, 8);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.DES.decrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString(CryptoJS.enc.Utf8);
}
};
var AES = {
encrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(16, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(0, 16);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.AES.encrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString();
},
decrypt: function (text, key, iv) {
var secretkey = (CryptoJS.MD5(key).toString()).substr(16, 16);
var secretiv = (CryptoJS.MD5(iv).toString()).substr(0, 16);
secretkey = CryptoJS.enc.Utf8.parse(secretkey);
secretiv = CryptoJS.enc.Utf8.parse(secretiv);
var result = CryptoJS.AES.decrypt(text, secretkey, {
iv: secretiv,
mode: CryptoJS.mode.CBC,
padding: CryptoJS.pad.Pkcs7
});
return result.toString(CryptoJS.enc.Utf8);
}
};
function getDecryptedData(data, AES_KEY_1, AES_IV_1, DES_KEY_1, DES_IV_1) {
data = AES.decrypt(data, AES_KEY_1, AES_IV_1);
data = DES.decrypt(data, DES_KEY_1, DES_IV_1);
data = BASE64.decrypt(data);
return data;
}
function ObjectSort(obj) {
var newObject = {};
Object.keys(obj).sort().map(function (key) {
newObject[key] = obj[key];
});
return newObject;
}
function getRequestParam(method, obj, appId) {
var clienttype = 'WEB';
var timestamp = new Date().getTime()
var param = {
appId: appId,
method: method,
timestamp: timestamp,
clienttype: clienttype,
object: obj,
secret: CryptoJS.MD5(appId + method + timestamp + clienttype + JSON.stringify(ObjectSort(obj))).toString()
};
param = BASE64.encrypt(JSON.stringify(param));
return param;
}
function getRequestAESParam(requestMethod, requestCity, appId, AES_KEY_2, AES_IV_2){
var param = getRequestParam(requestMethod, requestCity, appId);
return AES.encrypt(param, AES_KEY_2, AES_IV_2);
}
function getRequestDESParam(requestMethod, requestCity, appId, DES_KEY_2, DES_IV_2){
var param = getRequestParam(requestMethod, requestCity, appId);
return DES.encrypt(param, DES_KEY_2, DES_IV_2);
}
Python code examples matching various parameters in JS (encryption methods matching 8 key s, iv values, appId and param):
def get_key_iv_appid(decrypted_js):
"""
:param decrypted_js: Decrypted encrypt_xxxxxx.js
:return: Some parameters required by the request
"""
key_iv = re.findall(r'const.*?"(.*?)";', decrypted_js)
app_id = re.findall(r"var appId.*?'(.*?)';", decrypted_js)
request_data_name = re.findall(r"aqistudyapi.php.*?data.*?{(.*?):", decrypted_js, re.DOTALL)
# Judge whether param is encrypted by AES or DES or not
if "AES.encrypt(param" in decrypted_js:
request_param_encrypt = "AES"
elif "DES.encrypt(param" in decrypted_js:
request_param_encrypt = "DES"
else:
request_param_encrypt = "NO"
key_iv_appid = {
# The positions of key and iv are the same as those in the original js
"aes_key_1": key_iv[0],
"aes_iv_1": key_iv[1],
"aes_key_2": key_iv[2],
"aes_iv_2": key_iv[3],
"des_key_1": key_iv[4],
"des_iv_1": key_iv[5],
"des_key_2": key_iv[6],
"des_iv_2": key_iv[7],
"app_id": app_id[0],
# The key name of the data to send the request
"request_data_name": request_data_name[0].strip(),
# What kind of encryption is required to send the requested data value
"request_param_encrypt": request_param_encrypt
}
# print(key_iv_appid)
return key_iv_appid
Python code example of sending request and decrypting return value (take Beijing as an example):
def get_data(key_iv_appid):
"""
:param key_iv_appid: get_key_iv_appid() Method
"""
request_method = "GETDATA"
request_city = {"city": "Beijing"}
with open('main.js', 'r', encoding='utf-8') as f:
execjs_ = execjs.compile(f.read())
# Call different methods according to different encryption methods to obtain the param parameters of the request for encryption
request_param_encrypt = key_iv_appid["request_param_encrypt"]
if request_param_encrypt == "AES":
param = execjs_.call(
'getRequestAESParam', request_method, request_city,
key_iv_appid["app_id"], key_iv_appid["aes_key_2"], key_iv_appid["aes_iv_2"]
)
elif request_param_encrypt == "DES":
param = execjs_.call(
'getRequestDESParam', request_method, request_city,
key_iv_appid["app_id"], key_iv_appid["des_key_2"], key_iv_appid["des_iv_2"]
)
else:
param = execjs_.call('getRequestParam', request_method, request_city, key_iv_appid["app_id"])
data = {
key_iv_appid["request_data_name"]: param
}
response = requests.post(url=aqistudy_api, headers=headers, data=data).text
# print(response)
# Decrypt the obtained encrypted data
decrypted_data = execjs_.call(
'getDecryptedData', response,
key_iv_appid["aes_key_1"], key_iv_appid["aes_iv_1"],
key_iv_appid["des_key_1"], key_iv_appid["des_iv_1"]
)
print(json.loads(decrypted_data))
Run the result, successfully request and decrypt the return value:
