Programmer Think

statement All contents in this article are for learning and communication only. The packet capturing content, sensitive website and data interface have been desensitized. It is strictly prohibited to use them for commercial and illegal purposes, otherwise all the consequences have nothing to do with the author. If there is infringement, pl ...

Don't stay after the wind and the moon. The spring mountain is at the end of Pingwu. Finally have time to update my blog!! This time, let's climb Baidu Index. 1, Web page analysis We take reptile as the keyword to analyze Baidu Index Then, F12 developer mode, refresh, and click Network - > XHR - > index? area=0&word=... -> ...

1, Font anti crawl Introduction of font anti pickpocketing based on the case of starting point Chinese network Requirements: https://www.qidian.com/rank/yuepiao/ Get the title of the book and the number of monthly tickets ranked in the monthly ticket list of the starting point Chinese website Through packet capturing, we can find that the bo ...

26 data analysis cases - the fifth stop: data collection based on the Scrapy architecture Case environment Python: Python 3.x； Data description title: Course titleimage_url: Title picture address.properties: course nature.Stage: course stage.enrollment: number of course applicants. Data package Link: https://pan.baidu.com/s/1-DUUUAOfpC4G ...

xpath review Import third-party libraries, make requests to web pages, get html files, load html files into elementary objects and load them into tree s, then you can use the xpath method, which is an indeterminate path. By passing in the determined path as a string, you can find elements based on the path. Requirements: Remove the text and cl ...

Because the course assignment needs to carry out some analysis on NLP, and there is no particularly useful code on the Internet, so I just write a crawler myself, which can crawl the microblog content, comment content and microblog publisher related information according to the topic name. At present, there is no special problem in the author t ...

statement All contents in this article are for learning and communication only. The packet capturing content, sensitive website and data interface have been desensitized. It is strictly prohibited to use them for commercial and illegal purposes, otherwise all the consequences have nothing to do with the author. If there is infringement, plea ...

python crawler, crawling with scratch After learning the crawler for a period of time, I was ready to do a crawler exercise to consolidate it, so I chose the daily fund to crawl the data, and the problems and solutions encountered were recorded as follows. Attach code address: https://github.com/Marmot01/python-scrapy- Crawling idea I Analys ...

This blog is a supplement to the knowledge of the sweep selector. Sweep selector The scratch framework has its own data extraction mechanism. The related content is called selector seletors, which can select the specified part in HTML through XPath and CSS expressions. The sweep selector is implemented based on the parsel library, which is a ...

0x00 preparation Make up day accountPython 3 running environmentThird party libraries such as requests 0x01 process analysis Check the corresponding URL s of exclusive SRC, enterprise SRC and public SRC respectively, and it is found that there is no change. It is preliminarily judged that the website uses Ajax, that is, asynchronous JavaScri ...