[Python black technology] common user_agent browser head crawler simulation user (nanny level graphics + implementation code)

Posted by fellow21 on Thu, 03 Feb 2022 01:50:39 +0100

Welcome to pay attention "Python black technology" series , continuously updating
Welcome to pay attention "Python black technology" series , continuously updating

If the crawler doesn't use the user agent browser header, many websites can't climb basically,

Using the browser head to disguise the real user crawler is a basic knowledge of crawler anti crawling. Here are some commonly used users_ Agent for your reference.

1. Common browser request header user agent

This section refers to the blog https://blog.csdn.net/mouday/article/details/80182397

user_agent = [
    "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
    "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
    "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0",
    "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; rv:11.0) like Gecko",
    "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)",
    "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
    "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
    "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
    "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon 2.0)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; TencentTraveler 4.0)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser)",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
    "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
    "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
    "Mozilla/5.0 (iPad; U; CPU OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
    "Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",
    "MQQBrowser/26 Mozilla/5.0 (Linux; U; Android 2.3.7; zh-cn; MB200 Build/GRJ22; CyanogenMod-7) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",
    "Opera/9.80 (Android 2.3.4; Linux; Opera Mobi/build-1107180945; U; en-GB) Presto/2.8.149 Version/11.10",
    "Mozilla/5.0 (Linux; U; Android 3.0; en-us; Xoom Build/HRI39) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13",
    "Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.337 Mobile Safari/534.1+",
    "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0",
    "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124",
    "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)",
    "UCWEB7.0.2.37/28/999",
    "NOKIA5700/ UCWEB7.0.2.37/28/999",
    "Openwave/ UCWEB7.0.2.37/28/999",
    "Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999",
    # iPhone 6: 
	"Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25",
]

2. Randomly obtain a browser header

In practice, it is not good for the same user (browser header) to visit the website many times in a short time. Random multiple users (browser header) should be used to visit.

# Get a random request header
def get_random_user_agent():
    user_agent = [
        "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
        "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50",
        "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0",
        "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; InfoPath.3; rv:11.0) like Gecko",
        "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)",
        "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
        "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)",
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
        "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
        "Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11",
        "Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11",
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon 2.0)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; TencentTraveler 4.0)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; The World)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; SE 2.X MetaSr 1.0; SE 2.X MetaSr 1.0; .NET CLR 2.0.50727; SE 2.X MetaSr 1.0)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser)",
        "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
        "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
        "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
        "Mozilla/5.0 (iPad; U; CPU OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5",
        "Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",
        "MQQBrowser/26 Mozilla/5.0 (Linux; U; Android 2.3.7; zh-cn; MB200 Build/GRJ22; CyanogenMod-7) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1",
        "Opera/9.80 (Android 2.3.4; Linux; Opera Mobi/build-1107180945; U; en-GB) Presto/2.8.149 Version/11.10",
        "Mozilla/5.0 (Linux; U; Android 3.0; en-us; Xoom Build/HRI39) AppleWebKit/534.13 (KHTML, like Gecko) Version/4.0 Safari/534.13",
        "Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.1+ (KHTML, like Gecko) Version/6.0.0.337 Mobile Safari/534.1+",
        "Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.70 Safari/534.6 TouchPad/1.0",
        "Mozilla/5.0 (SymbianOS/9.4; Series60/5.0 NokiaN97-1/20.0.019; Profile/MIDP-2.1 Configuration/CLDC-1.1) AppleWebKit/525 (KHTML, like Gecko) BrowserNG/7.1.18124",
        "Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0; HTC; Titan)",
        "UCWEB7.0.2.37/28/999",
        "NOKIA5700/ UCWEB7.0.2.37/28/999",
        "Openwave/ UCWEB7.0.2.37/28/999",
        "Mozilla/4.0 (compatible; MSIE 6.0; ) Opera/UCWEB7.0.2.37/28/999",
        # iPhone 6: 
        "Mozilla/6.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/8.0 Mobile/10A5376e Safari/8536.25",

    ]
    return random.choice(user_agent)

Test code

# Printed random browser header
print(get_random_user_agent())

3. About user_ Introduction of agent knowledge

This section refers to the blog https://blog.csdn.net/u012175089/article/details/61199238

1, Basic knowledge:

User agent of HttpHeader
The Chinese name of UserAgent is user agent. It is a part of Http protocol and belongs to the header domain. UserAgent is also referred to as UA. It is a special string header, which is an identification that provides information such as the type and version of the browser you use, the operating system and version, the browser kernel, and so on. Through this logo, the website visited by users can display different typesetting, so as to provide users with better experience or information statistics; For example, accessing Google by mobile phone is different from accessing by computer. These are judged by Google according to the UA of visitors. UA can be disguised.
Standard format of UA string of browser: browser identification (operating system identification; encryption level identification; browser language) rendering engine identification version information. But each browser is different.
String description:

1. Browser identity

For the purposes of compatibility and promotion, many browsers have the same logo, so the browser logo does not indicate the real version of the browser. The real version information can be found at the end of the UA string.

2. Operating system identification

FreeBSD X11;FreeBSD(version no.)i386
X11;FreeBSD(version no.)AMD64
Linux X11;Linux ppc
X11;Linux ppc64
X11;Linux i686
X11;Linux x86_64
Mac Macintosh;PPC Mac OS X
Macintosh;Intel Mac OS X
Solaris X11;SunOS i86pc
X11;SunOs sun4u
Windows NT 6.1 corresponds to Windows 7
Windows NT 6.0 corresponds to windows Vista
Windows NT 5.2 corresponds to windows 2003
Windows NT 5.1 corresponds to windows xp
Windows NT 5.0 corresponds to windows 2000
Windows ME
Windows 98

3. Encryption level identification

N: Indicates no secure encryption
1: I ndicates weak security encryption
U: Indicates strong security encryption

4. Browser language

The language specified in Preferences > General > language

5. Rendering engine

The mainstream rendering engines used by the display browser include Gecko, WebKit, KHTML, Presto, Trident, Tasman, etc. the format is: rendering engine / version information

6. Version information

Display browser version information in the format of real browser version

Browser user agent details

PC end:
safari5.1–MAC
User-Agent:Mozilla/5.0(Macintosh;U;IntelMacOSX10_6_8;en-us)AppleWebKit/534.50(KHTML,likeGecko)Version/5.1Safari/534.50

safari5.1–Windows
User-Agent:Mozilla/5.0(Windows;U;WindowsNT6.1;en-us)AppleWebKit/534.50(KHTML,likeGecko)Version/5.1Safari/534.50

IE9.0
User-Agent:Mozilla/5.0(compatible;MSIE9.0;WindowsNT6.1;Trident/5.0;

IE8.0
User-Agent:Mozilla/4.0(compatible;MSIE8.0;WindowsNT6.0;Trident/4.0)

IE7.0
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT6.0)

IE6.0
User-Agent:Mozilla/4.0(compatible;MSIE6.0;WindowsNT5.1)

Firefox4.0.1–MAC
User-Agent:Mozilla/5.0(Macintosh;IntelMacOSX10.6;rv:2.0.1)Gecko/20100101Firefox/4.0.1

Firefox4.0.1–Windows
User-Agent:Mozilla/5.0(WindowsNT6.1;rv:2.0.1)Gecko/20100101Firefox/4.0.1

Opera11.11–MAC
User-Agent:Opera/9.80(Macintosh;IntelMacOSX10.6.8;U;en)Presto/2.8.131Version/11.11

Opera11.11–Windows
User-Agent:Opera/9.80(WindowsNT6.1;U;en)Presto/2.8.131Version/11.11

Chrome17.0–MAC
User-Agent:Mozilla/5.0(Macintosh;IntelMacOSX10_7_0)AppleWebKit/535.11(KHTML,likeGecko)Chrome/17.0.963.56Safari/535.11

Maxthon
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;Maxthon2.0)

Tencent TT
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;TencentTraveler4.0)

The world 2 x
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1)

TheWorld 3 x
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;TheWorld)

Sogou browser 1 x
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;Trident/4.0;SE2.XMetaSr1.0;SE2.XMetaSr1.0;.NETCLR2.0.50727;SE2.XMetaSr1.0)

360 browser
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;360SE)

Avant
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1;AvantBrowser)

GreenBrowser
User-Agent:Mozilla/4.0(compatible;MSIE7.0;WindowsNT5.1)

Mobile device end:

safariiOS4.33–iPhone
User-Agent:Mozilla/5.0(iPhone;U;CPUiPhoneOS4_3_3likeMacOSX;en-us)AppleWebKit/533.17.9(KHTML,likeGecko)Version/5.0.2Mobile/8J2Safari/6533.18.5

safariiOS4.33–iPodTouch
User-Agent:Mozilla/5.0(iPod;U;CPUiPhoneOS4_3_3likeMacOSX;en-us)AppleWebKit/533.17.9(KHTML,likeGecko)Version/5.0.2Mobile/8J2Safari/6533.18.5

safariiOS4.33–iPad
User-Agent:Mozilla/5.0(iPad;U;CPUOS4_3_3likeMacOSX;en-us)AppleWebKit/533.17.9(KHTML,likeGecko)Version/5.0.2Mobile/8J2Safari/6533.18.5

AndroidN1
User-Agent:Mozilla/5.0(Linux;U;Android2.3.7;en-us;NexusOneBuild/FRF91)AppleWebKit/533.1(KHTML,likeGecko)Version/4.0MobileSafari/533.1

Android QQ browser Forandroid
User-Agent:MQQBrowser/26Mozilla/5.0(Linux;U;Android2.3.7;zh-cn;MB200Build/GRJ22;CyanogenMod-7)AppleWebKit/533.1(KHTML,likeGecko)Version/4.0MobileSafari/533.1

AndroidOperaMobile
User-Agent:Opera/9.80(Android2.3.4;Linux;OperaMobi/build-1107180945;U;en-GB)Presto/2.8.149Version/11.10

AndroidPadMotoXoom
User-Agent:Mozilla/5.0(Linux;U;Android3.0;en-us;XoomBuild/HRI39)AppleWebKit/534.13(KHTML,likeGecko)Version/4.0Safari/534.13

BlackBerry
User-Agent:Mozilla/5.0(BlackBerry;U;BlackBerry9800;en)AppleWebKit/534.1+(KHTML,likeGecko)Version/6.0.0.337MobileSafari/534.1+

WebOSHPTouchpad
User-Agent:Mozilla/5.0(hp-tablet;Linux;hpwOS/3.0.0;U;en-US)AppleWebKit/534.6(KHTML,likeGecko)wOSBrowser/233.70Safari/534.6TouchPad/1.0

NokiaN97
User-Agent:Mozilla/5.0(SymbianOS/9.4;Series60/5.0NokiaN97-1/20.0.019;Profile/MIDP-2.1Configuration/CLDC-1.1)AppleWebKit/525(KHTML,likeGecko)BrowserNG/7.1.18124

WindowsPhoneMango
User-Agent:Mozilla/5.0(compatible;MSIE9.0;WindowsPhoneOS7.5;Trident/5.0;IEMobile/9.0;HTC;Titan)

UC none
User-Agent:UCWEB7.0.2.37/28/999

UC standard
User-Agent:NOKIA5700/UCWEB7.0.2.37/28/999

UCOpenwave
User-Agent:Openwave/UCWEB7.0.2.37/28/999

UCOpera
User-Agent:Mozilla/4.0(compatible;MSIE6.0;)Opera/UCWEB7.0.2.37/28/999

2, Browser recognition

1. IE browser (take IE9.0 as an example)
PC side: user agent: Mozilla / 5.0 (compatible; msie9.0; windowsnt6.1; Trident / 5.0);
Mobile device: user agent: Mozilla / 5.0 (compatible; msie9.0; windowsphoneos7.5; Trident / 5.0; iemobile / 9.0; HTC; Titan)
Because roam, window of the world, 360 browser, Tencent browser, Sogou browser, avant and GreenBrowser all adopt IE kernel, Therefore, the criterion for IE browser to judge is the "MSIE" field, and the number behind the MSIE field is the version number, but at the same time, it also needs to judge that it does not include "Maxthon", "Theworld", "360SE", "TencentTraveler", "SE", "Avant" and other fields (there is no explicit identification in the green browser). Mobile devices also need to judge the IEMobile + version number.

2. 360 browser
PC side: user agent: Mozilla / 4.0 (compatible; msie7.0; windowsnt5.1; Trident / 4.0; InfoPath. 2;. Net4.0c;. Net4.0e;. Netclr2.0.50727; 360se)
Mobile device: None
The judgment standard of 360 browser is the "360SE" field, without version representation.

3. Sogou browser
PC side: user agent: Mozilla / 4.0 (compatible; msie7.0; windowsnt5.1; Trident / 4.0; se2.xmetasr1.0; se2.xmetasr1.0;. Netclr2.0.50727; se2. Xmetasr1.0)
Mobile device: None
The judgment criteria of Sogou browser are "Se" and "MetaSr", and the version number is the number after SE.

4,Chrome
PC side: Mozilla / 5.0 (Macintosh; Intel macosx10_7_0) applewebkit / 535.11 (KHTML, likeko) Chrome / 17.0.963.56safari/535.11
Mobile device: user agent: Mozilla / 5.0 (Linux; u; android2.2.1; zh CN; htc_wildfire_a3333build / frg83d) applewebkit / 533.1 (KHTML, likeko) version / 4.0mobilesafari / 533.1
The judgment standard of PC Chrome browser is the chrome field, and the number behind Chrome is the version number; The Chrome browser on the mobile terminal judges "android", "linux", "mobilesafari" and other fields, and the number after version is the version number.

5,Safari
PC side: user agent: Mozilla / 5.0 (Macintosh; u; Intel macosx10_6_8; en US) applewebkit / 534.50 (KHTML, likeko) version / 5.1safari / 534.50
Mobile device: user agent: Mozilla / 5.0 (iPhone; u; cpuiphoneos4_3_3likemacosx; en US) applewebkit / 533.17.9 (KHTML, likeko) version / 5.0.2mobile/8j2safari/6533.18.5
Since chrome and Nokia's series60browser also use WebKit kernel, Safari browser's judgment must be: it contains Safari field and does not contain chrome and other information. After confirmation, the number after "version /" is the version number. Under the above conditions, Safari browser on Mobile device contains Mobile field.

6. Tencent browser
PC side: user agent: Mozilla / 4.0 (compatible; msie7.0; windowsnt5.1; Trident / 4.0; tencenttraveler4.0;. Netclr2.0.50727)
Mobile device: user agent: mqqbrowser / 26mozilla / 5.0 (Linux; u; android2.3.7; zh CN; mb200build / grj22; cyanogenmod-7) applewebkit / 533.1 (KHTML, likeko) version / 4.0mobilesafari / 533.1
The judgment standard of Tencent browser is "Tencent traveler" or "QQBrowser", and the number behind Tencent traveler or QQBrowser is the version number.

7,Firefox
PC side: user agent: Mozilla / 5.0 (windowsnt6.1; RV: 2.0.1) gecko / 20100101firefox / 4.0.1
Mobile device: user agent: Mozilla / 5.0 (Android; Linux armv7l; RV: 5.0) gecko / Firefox / 5.0fennec / 5.0
Firefox is judged by the Firefox field, and the number after Firefox is the version number.

8,Theworld
PC side: user agent: Mozilla / 4.0 (compatible; msie7.0; windowsnt5.1; theworld)
Mobile device: None
The judgment standard of Theworld browser is the "Theworld" field, which is not marked with the version number.
Note: Theworld2 There is no "Theworld" field in the user agent of version X.

9. Roam
PC side: user agent: Mozilla / 4.0 (compatible; msie7.0; windowsnt5.1; maxthon2.0)
Mobile device: None
The judgment standard of roaming browser is "Maxthon", and the number after Maxthon is the version number.

10,Opera
PC side: user agent: Opera / 9.80 (windowsnt6.1; u; EN) Presto / 2.8.131version/11.11
Mobile device: user agent: Opera / 9.80 (Android 2.3.4; Linux; opera Mobi / adr-1107051709; u; zh CN) Presto / 2.8.149version/11.10
The criterion of opera browser is opera field, and the number after opera field is version number.

11. UC browser
UCWeb has multiple browsing modes, and the corresponding user agent is:
UC none
User-Agent:UCWEB7.0.2.37/28/999
UC standard
User-Agent:NOKIA5700/UCWEB7.0.2.37/28/999
UCOpenwave
User-Agent:Openwave/UCWEB7.0.2.37/28/999
UCOpera
User-Agent:Mozilla/4.0(compatible;MSIE6.0;)Opera/UCWEB7.0.2.37/28/999
The judgment standard of UC browser is the "UCWEB" field, and the number after UCWEB is the version number.
Operating system orientation
The orientation of the operating system depends on the user agent in the httpheader information transmitted by each browser when opening the page. For the description of the user agent, see the user agent of httpader.
For details of user agent, see details of browser user agent.

Let's understand the identification methods of different operating systems in user agent.
PC end:
FreeBSD X11;FreeBSD(version no.)i386
X11;FreeBSD(version no.)AMD64
Linux X11;Linux ppc
X11;Linux ppc64
X11;Linux i686
X11;Linux x86_64
Mac Macintosh;PPC Mac OS X
Macintosh;Intel Mac OS X
Solaris X11;SunOS i86pc
X11;SunOs sun4u
Windows NT 6.1 corresponds to Windows 7
Windows NT 6.0 corresponds to windows Vista
Windows NT 5.2 corresponds to windows 2003
Windows NT 5.1 corresponds to windows xp
Windows NT 5.0 corresponds to windows 2000
Windows ME
Windows 98

Mobile device end:
iOS iPhone OS 4_3_3 iPhone/iPod
OS 4_3_3 iPad
The number after Android 2.3.7 is the version number
WebOs hpwOS/3.0.0
SymbianOS SymbianOS/9.4
Windows Phone OS Windows Phone OS 7.5
BlackBerry BlackBerry

summary

If you like, give me a 👍， Pay attention! Share more interesting Python black technology!

Copyright notice:

If you find that you are far away from the @mzh original work, you must mark the original link for reprint

Copyright 2022 mzh

Crated: 2022-2-1

Welcome to pay attention "Python black technology" series , continuously updating
Welcome to pay attention "Python black technology" series , continuously updating
[Python installs a third-party library with a one-line command to permanently increase the speed]
[package exe with PyInstaller]
[one click download of Zhihu article pictures without login crawler (nanny graphic + implementation code)]
[lonely programmers chat with AI robot friends to relieve boredom (free interface + nanny level graphics + implementation code comments)]
[draw gif dynamic diagram with a few lines of code (nanny level picture and text + implementation code)]
[several lines of code to realize regular and cyclic screenshots of online class and save important knowledge points (nanny level graphics and text + implementation code)]
[common user_agent browser head crawler simulates users (nanny level graphics + implementation code)]
[more details]

Topics: Python crawler

Hot Topics

©2024 Programmer Think