The original website Site statistics page I need bloggers to upload statistics manually every month. It's very troublesome and easy to kill people until I see it fox The statistics page of is a bright spot in front of me, so I went to learn from it. Note that most of the codes in this paper are from fox The hand of the big man. In addition, if you also want to use GitHubactions to update regularly and automatically, please refer to Above After completing the integrated deployment, it is said that the integrated deployment is really delicious. You can do a lot of things with actions and write articles online with hpp. You can enter the text without saying much.
Brief analysis of principle
Obtain the token of Baidu statistics open api, download and save the statistical data returned by Baidu statistics api as a json file using python, and then use JavaScript to read and process the json data, and pass the processed data through ecarts JS is drawn as a chart.
Get token and site_id
Before that, I believe you have completed the registration of Baidu statistics and will be able to use Baidu statistics normally. About obtaining token and site_id, you can refer to Baidu statistical document perhaps Blogger Eurkon's tutorial It's all very clear. I won't go into details. It is worth noting that the token obtained is valid for one month, and the refresh needs to be reused every other month_ The invalid link token can be used to update the token directly.
Download files using python
Through 6 links, we can obtain: daily access statistics, access map data, monthly access statistics, source classification statistics, search engine access statistics and external link access statistics within a year; Download and save through python. In order to avoid cross domain problems, the saved json file needs to be in conjunction with census JS in the same directory, cross domain issues can be solved through This post Find out. I'll show the complete structure directory first, and you can create it in the same way.
source └── census └── index.md //Auxiliary statistics page rendering └── census.js //Processing data └──data └──get.py //Get saved data theme └──matery └──layout └──census.ejs//Render as statistics page
Create a new get according to the above directory Py file and write the following contents in it:
import requests import time, datetime # Start date of Statistics start_date = '20201101' date = datetime.datetime.now() # End date of Statistics end_date = str(date.year) + (str(date.month) if date.month > 9 else ('0' + str(date.month))) + (str(date.day) if date.day > 9 else ('0' + str(date.day))) # token and siteid #It needs to be modified to your token and siteid access_token = '121.*******' site_id = '16*******' # Baidu statistics API dataUrl = 'https://openapi.baidu.com/rest/2.0/tongji/report/getData?access_token=' + access_token + '&site_id=' + site_id # Count the number of visits PV fill in 'pv_count ', count the number of visitors, and fill in' visitor '_ Count ', choose one from two metrics = 'pv_count' ips = 'visitor_count' def downFile(url, fileName, prefix=''): print('downloading :', url) down_res = requests.get(url) with open(prefix+fileName, 'wb') as f: f.write(down_res.content) print('writing :', prefix+fileName) # Visitor Map downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + metrics + '&method=visit/district/a', 'map.json') # Access trends downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + metrics + '&method=trend/time/a&gran=month', 'pv.json') downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + ips + '&method=trend/time/a&gran=month', 'uv.json') # Access source downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + metrics + '&method=source/all/a', 'sources.json') ## Search Engines downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + metrics + '&method=source/engine/a', 'engine.json') ## External links downFile(dataUrl + '&start_date=' + start_date + '&end_date=' + end_date + '&metrics=' + metrics + '&method=source/link/a', 'link.json') # Access calendar ''' To access the calendar, you need to obtain the data within a year, which is a little more than 52 weeks based on 365 days in a year. Therefore, there are 52 complete rows in front. The acquisition method is only through the start date and year-1 that will do Then there's row 53, python Medium date.weekday()The obtained day of the week is 0 corresponding to Monday, so it is passed(date.weekday()+1)%7 You can convert to Sunday corresponding to 0 So on the basis of 52 weeks, subtract the number of weeks to get a new one start_date ''' date = datetime.datetime(date.year-1, date.month, date.day) date = datetime.datetime.fromtimestamp(date.timestamp()-3600*24*((date.weekday()+1)%7)) start_date = str(date.year) + (str(date.month) if date.month > 9 else ('0' + str(date.month))) + (str(date.day) if date.day > 9 else ('0' + str(date.day))) downFile(dataUrl + '&method=overview/getTimeTrendRpt' + '&metrics=' + metrics + '&start_date=' + start_date + '&end_date=' + end_date, 'calendar.json') downFile(dataUrl + '&method=overview/getTimeTrendRpt' + '&metrics=' + ips + '&start_date=' + start_date + '&end_date=' + end_date, 'ipcalendar.json')
Local test: if you have a python environment installed locally, you can directly store the cd to get Py directory Python get Py, prompt the lack of library installation. If you haven't completed the integrated deployment of GitHubactions, you can also update it semi automatically on a regular basis through this method, but it's recommended to make the integrated deployment. It's really delicious.
Create new md and ejs files
As shown in the previous structure diagram, create a new index MD and census EJS file.
In index Write the following in MD:
--- title: census date: 2020-10-31 10:11:28 type: "census" layout: "census" ---
At census Write the following in EJS:
<%- partial('_partial/bg-cover') %> <style> .journal { padding: 12px; border: 1px dashed #e6e6e6; color: #969696; position: relative; display: inline-block; width: 95%; background: #fbfbfb50; border-radius: 10px; font-size: 16px; margin: 12px auto; } </style> <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/echarts@5.1.1/dist/echarts.min.js"></script> <script src="https://cdn.jsdelivr.net/npm/echarts@4.7.0/map/js/china.js"></script> <main class="content"> <div class="container chip-container"> <div class="card"> <div class="card-content"> <div class="tag-title center-align"> <div class="journal"> <div class="title center-align">" "Site statistics"</div> " Since 2020-11-14 Automatically update every 12 hours from " </div> </div> <div id="calendar_container" style="min-height: 200px; -webkit-tap-highlight-color: transparent; user-select: none; position: relative"></div> <div id="uv_container" style="min-width: 250px; height: 400px; margin-top: 50px; -webkit-tap-highlight-color: transparent; user-select: none; position: relative" ></div> <div id="pv_container" style="min-width: 250px; height: 400px; margin-top: 50px; -webkit-tap-highlight-color: transparent; user-select: none; position: relative" ></div> <div id="map_container" style="min-width: 250px; height: 400px; margin-top: 50px; -webkit-tap-highlight-color: transparent; user-select: none; position: relative" ></div> <div id="sources_container" style="min-width: 250px; height: 400px; margin-top: 50px; -webkit-tap-highlight-color: transparent; user-select: none; position: relative" ></div> <script src="/census/census.js"></script> </div> </div> </div> </main>
calendar_container is the container of visitor calendar chart, uv_container is the container of the statistical chart of the number of visitors to the site, pv_container is the container of the site traffic statistics chart, map_container is the container for the site to access the location map, sources_container is the container that analyzes the pie chart for the source of site visitors.
Create a license JS file
Create a license as shown in the previous structure diagram JS file writes the following contents:
var metrics = 'pv_count' // Count the number of visits PV fill in 'pv_count ', count the number of visitors, and fill in' visitor '_ Count ', choose one from two var metricsName = (metrics === 'pv_count' ? 'Number of visits' : (metrics === 'visitor_count' ? 'Number of visitors' : '')) function generatePieces(maxValue, colorBox) { var pieces = []; var quotient = 1; var temp = {'lt': 1, 'label': '0', 'color': colorBox[0]}; pieces.push(temp); if (maxValue && maxValue >= 10) { quotient = Math.floor(maxValue / 10)+1; for (var i = 1; i <= 10; i++) { var temp = {}; if (i == 1) temp.gte = 1; else temp.gte = quotient * (i - 1); temp.lte = quotient * i; temp.color = colorBox[i]; pieces.push(temp); } } return JSON.stringify(pieces); } var append_div_visitcalendar = (parent, text) => { if (parent !== null) { if (typeof text === 'string') { var temp = document.createElement('div'); temp.innerHTML = text; var frag = document.createDocumentFragment(); while (temp.firstChild) { frag.appendChild(temp.firstChild) } parent.appendChild(frag) } else { parent.appendChild(text) } } }; function calChart () { let script = document.createElement("script") fetch('/census/data/ipcalendar.json?date'+new Date()).then(data => data.json()).then(data => { let date_arr = data.result.items[0]; let value_arr = data.result.items[1]; let calArr = []; let maxValue = 0, total = 0, weekdatacore = 0, thisweekdatacore = 0; let colorBox = ['#EBEDF0', '#90EE90', '#98FB98', '#32CD32', '#00FF00', '#7FFF00', '#3CB371', '#2E8B57', '#228B22', '#008000', ' #006400']; for (let i = 0; i < date_arr.length; i++) { calArr.push([date_arr[i][0], value_arr[i][0] === '--' ? 0 : value_arr[i][0]] ); maxValue = value_arr[i][0] > maxValue ? value_arr[i][0] : maxValue ; total += value_arr[i][0] === '--' ? 0 : value_arr[i][0]; } for (let i = date_arr.length-1; i >= date_arr.length-7; i--) weekdatacore += value_arr[i][0] === '--' ? 0 : value_arr[i][0]; for (let i = date_arr.length-1; i >= date_arr.length-30; i--) thisweekdatacore += value_arr[i][0] === '--' ? 0 : value_arr[i][0]; let calArrJson = JSON.stringify(calArr); script.innerHTML = ` var calChart = echarts.init(document.getElementById("calendar_container")); var option = { title: { text: 'Access calendar' }, tooltip: { padding: 10, backgroundColor: '#555', borderColor: '#777', borderWidth: 1, textStyle: { color: '#fff' }, formatter: function (obj) { var value = obj.value; return '<div style="font-size: 14px;">' + value[0] + ': ' + value[1] + '</div>'; } }, visualMap: { show: false, showLabel: true, min: 0, max: ${maxValue}, type: 'piecewise', orient: 'horizontal', left: 'center', bottom: 0, pieces: ${generatePieces(maxValue, colorBox)} }, calendar: [{ left: 'center', range: ['${date_arr[0]}', '${date_arr[date_arr.length-1]}'], cellSize: [14, 14], splitLine: { show: false }, itemStyle: { color: '#ebedf0', borderColor: '#fff', borderWidth: 2 }, yearLabel: { show: false }, monthLabel: { nameMap: 'cn', fontSize: 11 }, dayLabel: { formatter: '{start} 1st', nameMap: 'cn', fontSize: 11 } }], series: [{ type: 'heatmap', coordinateSystem: 'calendar', calendarIndex: 0, data: ${calArrJson}, }] }; calChart.setOption(option);`; let style = '<style>.number{font-family: sans-serif, Arial;margin-top: 10px;text-align:center;width:100%;padding:10px;margin:0 auto;}.contrib-column{text-align:center;border-left:1px solid #ddd;border-top:1px solid #ddd;}.contrib-column-first{border-left:0;}.table-column{padding:10px;display:table-cell;flex:1;vertical-align:top;}.contrib-number{font-weight:400;line-height:1.3em;font-size:24px;display:block;}.left.text-muted{float:left;margin-left:9px;color:#767676;}.left.text-muted a{color:#4078c0;text-decoration:none;}.left.text-muted a:hover{text-decoration:underline;}h2.f4.text-normal.mb-3{display:none;}.float-left.text-gray{float:left;}.position-relative{width:100%;}@media screen and (max-width:650px){.contrib-column{display:none}}</style>'; style = '<div style="display:flex;width:100%" class="number"><div class="contrib-column contrib-column-first table-column"><span class="text-muted">Visits in the past year</span><span class="contrib-number">' + total + '</span><span class="text-muted">' + date_arr[0][0] + ' - ' + date_arr[date_arr.length-1][0] + '</span></div><div class="contrib-column table-column"><span class="text-muted">Visit in the last 30 days</span><span class="contrib-number">' + thisweekdatacore + '</span><span class="text-muted">' + date_arr[date_arr.length-30][0] + ' - ' + date_arr[date_arr.length-1][0] + '</span></div><div class="contrib-column table-column"><span class="text-muted">Visit in the last 7 days</span><span class="contrib-number">' + weekdatacore + '</span><span class="text-muted">' + date_arr[date_arr.length-7][0] + ' - ' + date_arr[date_arr.length-1][0] + '</span></div></div>' + style; document.getElementById("calendar_container").after(script); append_div_visitcalendar(calendar_container, style); }).catch(function (error) { console.log(error); }); } function get_year(s) { return parseInt(s.substr(0, 4)) } function get_month(s) { return parseInt(s.substr(5, 2)) } // Views function pvChart () { let script = document.createElement("script") fetch('/census/data/pv.json?date'+new Date()).then(data => data.json()).then(data => { let date = new Date(); let monthValueArr = {}; let monthName = data.result.items[0]; let monthValue = data.result.items[1]; for (let i =2020; i <= date.getFullYear(); i++) monthValueArr[String(i)] = [ , , , , , , , , , , , ]; monthValueArr for (let i = 0; i < monthName.length; i++) { let year = get_year(monthName[i][0]); let month = get_month(monthName[i][0]); monthValueArr[String(year)][String(month-1)] = monthValue[i][0]; } script.innerHTML = ` var pvChart = echarts.init(document.getElementById('pv_container'), 'light'); var pvOption = { color: ['#01C2F9', '#18D070', '#d223e7', '#3F77FE'], title: { text: 'Site traffic statistics', subtext: 'data sources : Baidu Statistics (since 2020)/11/14 (start Statistics)', textStyle: { color: '#504b4d', } }, legend: { data: ['2020 Annual visits', '2021 Annual visits'], //Year of revision bottom: 0, left: 'center', textStyle: { color: '#504b4d', } }, tooltip: { trigger: 'axis' }, toolbox: { show: true, feature: { mark: { show: true }, magicType: { show: true, type: ['line', 'bar', 'stack', 'tiled'] }, restore: { show: true }, saveAsImage: { show: true } } }, calculable: true, xAxis: [{ type: 'category', boundaryGap: false, data: ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'], axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } } }], yAxis: [{ type: 'value', axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } }, axisTick: { show: true }, axisLine: { show: true, lineStyle: { color: '#4c4948'} } }], series: [{ name: '2020 Annual visits', type: 'line', stack: 'total', data: [${monthValueArr["2020"]}], axisLabel: { formatter: '{value}', //The data sequence corresponding to the first year corresponds to the month textStyle: { color: '#929298' } } }, { name: '2021 Annual visits', type: 'line', stack: 'total', data: [${monthValueArr["2021"]}], //The data sequence corresponding to the second year corresponds to the month axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } } }] }; pvChart.setOption(pvOption); window.addEventListener("resize", () => { pvChart.resize(); });` document.getElementById('pv_container').after(script); }).catch(function (error) { console.log(error); }); } //Number of visitors function uvChart () { let script = document.createElement("script") fetch('/census/data/uv.json?date'+new Date()).then(data => data.json()).then(data => { let date = new Date(); let monthValueArr = {}; let monthName = data.result.items[0]; let monthValue = data.result.items[1]; for (let i =2020; i <= date.getFullYear(); i++) monthValueArr[String(i)] = [ , , , , , , , , , , , ]; monthValueArr for (let i = 0; i < monthName.length; i++) { let year = get_year(monthName[i][0]); let month = get_month(monthName[i][0]); monthValueArr[String(year)][String(month-1)] = monthValue[i][0]; } script.innerHTML = ` var uvChart = echarts.init(document.getElementById('uv_container'), 'light'); var uvOption = { color: ['#d223e7', '#3F77FE', '#01C2F9', '#18D070'], title: { text: 'Statistics of site visitors', subtext: 'data sources : Baidu Statistics (since 2020)/11/14 (start Statistics)', textStyle: { color: '#504b4d', } }, tooltip: { trigger: 'axis' }, legend: { data: ['2020 Annual visitors', '2021 Annual visitors'], bottom: 0, left: 'center', textStyle: { color: '#504b4d', } }, //Year of revision toolbox: { show: true, feature: { mark: { show: true }, magicType: { show: true, type: ['line', 'bar', 'stack', 'tiled'] }, restore: { show: true }, saveAsImage: { show: true } } }, calculable: true, xAxis: [{ type: 'category', boundaryGap: false, data: ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'], axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } } }], yAxis: [{ type: 'value', axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } }, axisTick: { show: true }, axisLine: { show: true, lineStyle: { color: '#4c4948'} } }], series: [{ name: '2020 Annual visitors', type: 'line', smooth: true, itemStyle: { normal: { areaStyle: { type: 'default' } } }, data: [${monthValueArr["2020"]}], //The data sequence corresponding to the first year corresponds to the month axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } }, }, { name: '2021 Annual visitors', type: 'line', smooth: true, itemStyle: { normal: { areaStyle: { type: 'default' } } }, data: [${monthValueArr["2021"]}], //The data sequence corresponding to the second year corresponds to the month axisLabel: { formatter: '{value}', textStyle: { color: '#929298' } }, }] }; uvChart.setOption(uvOption); window.addEventListener("resize", () => { uvChart.resize(); });` document.getElementById('uv_container').after(script); }).catch(function (error) { console.log(error); }); } // Access map function mapChart () { let script = document.createElement("script") fetch('/census/data/map.json?date'+new Date()).then(data => data.json()).then(data => { let mapName = data.result.items[0] let mapValue = data.result.items[1] let mapArr = [] let max = mapValue[0][0] for (let i = 0; i < mapName.length; i++) { mapArr.push({ name: mapName[i][0].name, value: mapValue[i][0] }) } let mapArrJson = JSON.stringify(mapArr) script.innerHTML = ` var mapChart = echarts.init(document.getElementById('map_container'), 'light'); var mapOption = { title: { text: 'Visit location' }, tooltip: { trigger: 'item' }, visualMap: { min: 0, max: ${max}, left: 'left', top: 'bottom', text: ['high','low'], color: ['#1E90FF', '#AAFAFA'], calculable: true }, series: [{ name: '${metricsName}', type: 'map', mapType: 'china', showLegendSymbol: false, label: { emphasis: { show: false } }, itemStyle: { normal: { areaColor: 'rgba(255, 255, 255, 0.1)', borderColor: '#121212' }, emphasis: { areaColor: 'gold' } }, data: ${mapArrJson} }] }; mapChart.setOption(mapOption); window.addEventListener("resize", () => { mapChart.resize(); });` document.getElementById('map_container').after(script); }).catch(function (error) { console.log(error); }); } // Access source function sourcesChart () { let script = document.createElement("script"); var innerHTML = ''; var link = 0, direct = 0, search = 0; fetch('/census/data/sources.json?date'+new Date()).then(data => data.json()).then(data => { let sourcesName = data.result.items[0]; let sourcesValue = data.result.items[1]; let sourcesArr = []; for (let i = 0; i < sourcesName.length; i++) sourcesArr.push({ name: sourcesName[i][0].name, value: sourcesValue[i][0] }); link = sourcesArr[1]['value'] ; search = sourcesArr[2]['value'] ; direct = sourcesArr[0]['value'] ; innerHTML += ` var sourcesChart = echarts.init(document.getElementById('sources_container'), 'light'); var sourcesOption = { title:{text:'Site visitor source statistics',itemGap:20,textStyle:{color:'#504b4d',}}, tooltip: { trigger: 'item', formatter: '{a} <br/>{b}: {c} ({d}%)' }, legend: { data: ['Direct', 'Outer chain', 'search', 'Baidu', 'Google', 'Bing', 'Github', 'Bound for/Ten year covenant'], y: 'bottom' }, series: [ { name: 'Source details', type: 'pie', radius: ['45%', '60%'], labelLine: { length: 30 }, label: { formatter: '{a|{a}}{abg|}\\n{hr|}\\n {b|{b}: }{c} {per|{d}%} ', backgroundColor: '#F6F8FC', borderColor: '#8C8D8E', borderWidth: 1, borderRadius: 4, rich: { a: { color: '#6E7079', lineHeight: 22, align: 'center' }, hr: { borderColor: '#8C8D8E', width: '100%', borderWidth: 1, height: 0 }, b: { color: '#4C5058', fontSize: 14, fontWeight: 'bold', lineHeight: 33 }, per: { color: '#fff', backgroundColor: '#4C5058', padding: [3, 4], borderRadius: 4 } } }, data: [`; }).catch(function (error) { console.log(error); }); fetch('/census/data/engine.json?date'+new Date()).then(data => data.json()).then(data => { let enginesName = data.result.items[0]; let enginesValue = data.result.items[1]; let enginesArr = []; for (let i = 0; i < enginesName.length; i++) enginesArr.push({ name: enginesName[i][0].name, value: enginesValue[i][0] }); innerHTML += ` {value: ${enginesArr[1]['value']}, name: 'Google'}, {value: ${enginesArr[0]['value']}, name: 'Baidu'},`; }).catch(function (error) { console.log(error); }); fetch('/census/data/link.json?date'+new Date()).then(data => data.json()).then(data => { let linksName = data.result.items[0]; let linksValue = data.result.items[1]; let linksArr = {}; for (let i = 0; i < linksName.length; i++) linksArr[linksName[i][0].name] = linksValue[i][0]; let sum = data.result.sum[0][0]; let bing = linksArr['http://cn.bing.com']+linksArr['http://www.bing.com']; let github = linksArr['http://github.com']; innerHTML += ` {value: ${bing}, name: 'Bing'}, {value: ${direct}, name: 'Direct'}, {value: ${github}, name: 'Github'}, {value: ${sum-bing-github}, name: 'Friend chain'} ] }, { name: 'Access source', type: 'pie', selectedMode: 'single', radius: [0, '30%'], label: { position: 'inner', fontSize: 14}, labelLine: { show: false }, data: [ {value: ${search+bing}, name: 'search', itemStyle: { color : 'green' }}, {value: ${direct}, name: 'Direct', itemStyle: { color : '#FFDB5C' }}, {value: ${link-bing}, name: 'Outer chain', itemStyle: { color : '#32C5E9' }} ] }, ] }; sourcesChart.setOption(sourcesOption); window.addEventListener("resize", () => { sourcesChart.resize(); });`; script.innerHTML = innerHTML; }).catch(function (error) { console.log(error); }); document.getElementById('sources_container').after(script); } if (document.getElementById("calendar_container")) calChart(); if (document.getElementById('map_container')) mapChart(); if (document.getElementById('uv_container')) uvChart(); if (document.getElementById('pv_container')) pvChart(); if (document.getElementById('sources_container')) sourcesChart();
Skip rendering due to license JS will be rendered under the source folder, such as > will be rendered as gt. Skip in the site yml_ Render: add skip rendering items as follows:
skip_render: - 'census/census.js' - 'census/data/**'
There is no need to add navigation in the topic yml. Here you can view it through the local preview of hexo s.
Configure GitHub Action
blog (repository) └── .github └── workflows └── getdata.yml
Create a new GetData according to the above directory YML if you have completed the integrated deployment of GitHubactions. Write the following:
name: Get Baidu statistics on: schedule: - cron: ' 0 0,6,12,18 * * *' jobs: deploy: runs-on: ubuntu-latest steps: - name: Check branch uses: actions/checkout@v2 with: ref: main #After October 2020, the default branch of the new warehouse in github will be changed to main. Please pay attention to the change - name: Set up Python #Install python uses: actions/setup-python@v1 with: python-version: 3.8 - name: get data run: | cd /home/runner/work/hexo/hexo/source/census/data/ pip install requests python get.py cd /home/runner/work/hexo/hexo git config --global user.name "Yours GitHub User name as mine brqs" git config --global user.email "Yours GitHub E.g. 3447851674@qq.com" git add . git commit -m "Baidu statistics upload" git push origin main
Upload to the warehouse. I set it to be automatically executed every six hours. You can also reduce the frequency.
reference material
These are the references of my article. Some codes in the article come from these articles. Of course, it is recommended that you read these articles to learn more about the site statistics chart.