Preface

Netease Cloud Music has always been the "altar" I yearn for. When I listen to the music and see the commentary from the heart, it's a flowing mountain. So today I'm going to grab some hot reviews of the song. And make a CI cloud to show, see what is the most impressive commentary relative to this song.

The advantage of making word clouds is intuitive and beautiful. I can't think of anything else.

Grasping Data

To make a word cloud, you have to have data first. So a little crawling skill is needed.

• Analysis of grabbing bags
• Encryption Information Processing
• Grasp the Contents of Hot Comments

Analysis of grabbing bags

Use the Chrome console. We can easily find the link where the review is located. As follows:

Now that the URL has been found, the next step is data grabbing. But after a simple attempt, we found that we could not get detailed information, but returned empty strings.
Looking at hreaders again, we found that the browser used POST to make requests. The specific fields are as follows:

Encryption Information Processing

Then, after my test, I can take these two data from the browser directly. But in order to really solve this encryption processing, we need to encrypt and decrypt only storage. On GitHub, Daniel analyzed the details of the encrypted fields of Netease Cloud Music. If you are interested, please refer to the link below.

I'll use this temporary method here, and it can be reused for different songs. We can verify it later.

Grasp the Contents of Hot Comments

The interface is obtained from the first step, and the data returned is a JSON string, which can be used with a little processing.

# coding: utf8

# @ Author: Guo Pu
# @ File: Hot Comments on Netease Cloud Music Songs.                                                                 
# @Time: 2017/4/26                                   
# @Contact: 1064319632@qq.com
# @blog: http://blog.csdn.net/marksinoberg
# @ Description: Getting Popular Comments

import requests
import json

def getcomments(musicid):
    url = 'http://music.163.com/weapi/v1/resource/comments/R_SO_4_{}?csrf_token=5594eaee83614ea8ca9017d85cd9d1b3'.format(musicid)
    payload = {
        'params': '4hmFbT9ZucQPTM8ly/UA60NYH1tpyzhHOx04qzjEh3hU1597xh7pBOjRILfbjNZHqzzGby5ExblBpOdDLJxOAk4hBVy5/XNwobA+JTFPiumSmVYBRFpizkWHgCGO+OWiuaNPVlmr9m8UI7tJv0+NJoLUy0D6jd+DnIgcVJlIQDmkvfHbQr/i9Sy+SNSt6Ltq',
        'encSecKey': 'a2c2e57baee7ca16598c9d027494f40fbd228f0288d48b304feec0c52497511e191f42dfc3e9040b9bb40a9857fa3f963c6a410b8a2a24eea02e66f3133fcb8dbfcb1d9a5d7ff1680c310a32f05db83ec920e64692a7803b2b5d7f99b14abf33cfa7edc3e57b1379648d25b3e4a9cab62c1b3a68a4d015abedcd1bb7e868b676'
    }
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36',
        'Referer': 'http://music.163.com/song?id={}'.format(musicid),
        'Host': 'music.163.com',
        'Origin': 'http://music.163.com'
    }

    response = requests.post(url=url, headers=headers, data=payload)
    data = json.loads(response.text)
    hotcomments = []
    for hotcomment in data['hotComments']:
        item = {
            'nickname': hotcomment['user']['nickname'],
            'content': hotcomment['content']
        }
        hotcomments.append(item)

    # Return to Hot Comments
    return [content['content'] for content in hotcomments]

if __name__ == '__main__':
    hot = getcomments(439915614)
    print(hot)

Look at the results.

['First love is very simple, with her good nearly a year, and finally take her home to meet her parents. During the meal, the mother kept picking up her dishes. "Come on, girl, and let him take you back when you've finished eating." My girlfriend may have misunderstood my mother's meaning, immediately put down the dishes and chopsticks, a face of grievance, "Uncle and aunt, you don't drive me away, I really like him, I will eat less later." Hearing his girlfriend say this, my heart instantaneously changed.', 'Li Yugang Choir has been launched, soprano Li Yugang, bass Li Yugang, tenor Li Yugang, bass Li Yugang[Laugh][Laugh][Laugh]', '2012Years, that year's freshman, military training, we met. Winter in a foreign country frostbite her first love, but I have no reason to give her a comforting hug. From then on, we were still friends, occasionally greeting each other, and then dispersed in the turbulent crowd. She had another him, and then withered.2015Year 3. We went for a walk together. I confessed to her under the cover of the night. She took my hand and cried, "You said it after all."', 'After graduating from university, I went to Beijing, and she stayed in her hometown. At first, she made a telephone call every day. Then she became fewer and fewer, and even cut off contact. Until two years later, I went home by train. After leaving the station, I saw her standing outside alone, her hair was blown disorderly by the wind. After seeing me, she did not move. I put down my luggage. She used to hold her in her arms, and she immediately burst into tears and said to me that I thought you didn't want me. From that moment on, I knew it was you all my life.', '「Just.」good「Jade」see「plum」', '96He left home and ran about in 1989.06Nian Xingguang Avenue lets the national audience know you.16On the 10th anniversary of your debut, you have spent ten years to prove to the world that you are not the immature Li Yugang who first appeared on the CCTV stage. You are Mr. Yu who has accumulated over ten years. You are the pioneer of spreading traditional culture. You are the dreamer who breaks away from shackles and pursues artistic breakthroughs.17Tenth Anniversary Concert[Kiss]', 'The chorus is poisonous. The studio was shown three times today.[Cry loudly]What the hell did you hear it for the first time? The second time I felt good. The third time I started to be poisoned. I picked up my cell phone and listened to the songs. I started to cycle the singles.[Cry loudly]', 'I used to lie in the same bed with a woman all night, not a lover. Nothing was done. Every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day, every day. I sang the song to her that night. After waking up in the morning, the intimacy index between the two sides increased geometrically to the previous one. n Secondary. Now, she's my woman. We'll get married in a few months. First love, thank you for meeting you', 'When I met him in the first year of high school, I fell in love with him at first sight. At the end of the exam, I went to see the exam number on his desk. His name was Mu Jiaming. I like his name. He went to play ball. No matter how far away I would run. He was sick. I ran to buy medicine. He ran in the sports meeting. I went to give him water and warm his stomach. I knew when I gave the injection. When he played nearby, he ran over and his cold worsened. The whole world knew that I loved him very much. The whole world knew that he didn't love me.', 'Acquaintance7Years, but together only then8Month, blame me me I did not have enough courage, I did not expect you will persist for me for so long. Thank God for letting me just meet you. It's your perseverance that makes us walk together finally. I hope you can leave a long memory in my life. I will cherish this hard-won encounter. In the future, let me give you the courage to go on, give me the opportunity to protect you, care for you, Liu Xiaoxian I love you...', 'In a freshman year, I came back early from vacation to sit in an Internet cafe and play Warcraft. The song was on demand. N Everywhere, there is a simple melody in my head.:Because I happened to meet you. In the evening, I hold my cell phone and say goodnight to each other. I dreamed of her in my dream. Now I just wake up and find the song's infinite cycle. Blessing all the people who are in love in other places to get married.[love]', 'We cried and we laughed.[Cry loudly]Open your mouth already full of tears, so I miss that time.', 'I used NetEasy Cloud music, the original trust to buy her mobile phone, to install NetEasy Cloud for her to log on to my account, but she does not use much.16year12month22She sent me away on the No. 1, and even though I'm not in touch with you anymore, I suddenly found out yesterday that there was one more song in my favorite music list, which is "Just Meet You".', 'Because I happened to meet you more than 140 days away from the college entrance examination and you became a classmate about a year ago. I didn't expect you secretly liked me. For a long time, the total score of our provincial mock exam was exactly the same. You said that we could go to the north to see the snow and hot pot. You said that I liked it. Professional is not bad. You say we are lucky, but ah, the most important sentence, why don't you say it?', 'That year you18，I20. I went to the hotel for an internship. I just met you after work. I don't believe in love at first sight, but you are the exception. Now, I25，you23. Thank you for coming to my world and never leaving. Next year, we're going to get married.']

Word Cloud

Wordcloud, a third-party library I use, can be installed using pip. There are very detailed and clear cases available for reference on the official website, and there are no repetitive descriptions here. Interested refer to the link below.
https://amueller.github.io/word_cloud/auto_examples/index.html

Now let's go directly to the code.

But neither the official website nor the other places spoke about the handling of Chinese. Then I looked and looked, and finally I found the solution, which is to specify the path of the font file on the construction method of WordCloud. In this way, we can solve the problem of Chinese random code. Specific settings are as follows:

WordCloud(random_state=1, font_path = r'C:/Users/Windows/fonts/simkai.ttf')

# coding: utf8

# @ Author: Guo Pu
# @ File: Word Cloud Test.py                                                                 
# @Time: 2017/4/26                                   
# @Contact: 1064319632@qq.com
# @blog: http://blog.csdn.net/marksinoberg
# @ Description: Word Cloud Testing
from wordcloud import WordCloud

from temp import Popular Comments on Netease Yun's Musical Songs as hot
text = " ".join(hot.getcomments(439915614))

wordcloud = WordCloud(random_state=1, font_path = r'C:/Users/Windows/fonts/simkai.ttf').generate(text)

import matplotlib.pyplot as plt
plt.figure()
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()

Word Cloud Operation Effect

Finally, compare the results of the operation.

summary

Finally, to review, this paper crawls the popular reviews of Netease Cloud Music, and produces the corresponding word clouds through the word cloud generator.

What can be expanded:

• Word cloud mask, which supports cloud generation in specific contexts
• Popular comments crawl in batches, just need to get the ID of the corresponding song.
• Encapsulate, extract interfaces, provide hot review services to the outside world, or word cloud generation services.
• ... ...