84 crawler-scrapy-redis source code analysis (dupefilter)

Responsible for the implementation of requst de-weighting, the implementation is very skillful, using the set data structure of redis. Note, however, that scheduler does not use the dupefilter key used to implement request s in this module, but uses ...

Posted by techmeister on Mon, 02 Sep 2019 06:30:14 +0200

Redis Learning Notes - Lua Scripts - Using Lua Scripts

Reference resources: http://www.redis.cn/commands/eval.html https://www.runoob.com/redis/redis-scripting.html Redis has supported Lua scripting since version 2.6.0. By embedding the Lua environment in the server, Redis clients can use Lua scripts t ...

Posted by harnacks on Mon, 02 Sep 2019 04:10:05 +0200

SpringBoot Integrated WebSocket+Socketjs Heart Reconnection Implementation

I. Basic concepts 1. Unicast: point-to-point, private chat 2. Multicast, also known as Multicast (Special Crowds): Multi-person chat, publish and subscribe 3. Broadcast (everyone): Game announcements, publishing and subscribing 2. Springboot Inte ...

Posted by JsusSalv on Fri, 30 Aug 2019 07:35:47 +0200

Using WxJava to implement website integration with WeChat login function, the core code should not exceed 10 lines

Recently, the website PC-side integrated WeChat scanner login, stepped on a number of pits, and recorded the implementation process and considerations. Table of Contents 1. Operational steps of WeChat Open Platform1. Create a Web Site Application2. Get AppID and AppSecret2. Development Guidelines3. Development of Actual Warfare1. pom.xm ...

Posted by tuneout on Thu, 29 Aug 2019 04:44:27 +0200

nginx seven-tier load balancing

[tcp] nginx seven-tier load balancing Overview of nginx load balancing When our Web servers are directly user-oriented, they often carry a large number of concurrent requests. It is difficult for a single server to load. I use multiple Web servers to form a cluster. The front-end uses Nginx load balancing to distribute requests to our back-end ...

Posted by mingmangat on Tue, 27 Aug 2019 15:43:34 +0200

Redis Environment Building - High Availability Cluster (HA)

In the production environment, Sentinel master-slave architecture is the most widely used Redis architecture, because a single point is prone to failures, and fragmented cluster is too complex. The author tries to build a Redis cluster architecture o ...

Posted by swon on Tue, 27 Aug 2019 10:19:15 +0200

Distributed Lock-Redission

Redission Distributed Lock brief introduction Redission is a Redis Official Network Distributed Solution Official website: https://redisson.org/ github: https://github.com/redisson/redisson#quick-start function   usedBy API   ...

Posted by marsooka on Sat, 24 Aug 2019 04:07:40 +0200

SSO Single Sign-on (Client) Based on Spring Security + OAuth2

1. Origin Why do you want to take the client out and write it separately? Bloggers also refer to a lot of single sign-on on the Internet, but they are basically the same and the same. In the client's own privilege checking and single exit are not p ...

Posted by Flukey on Thu, 22 Aug 2019 13:18:49 +0200

Distributed Construction of redis cluster Single Machine Pseudo-3 Master-3 Slave-3 Sentinel Cluster

Recently, the company introduced a micro-service framework, the previous redis 60G storage has been unable to meet the current 260G business needs, after some consideration to build the cluster. In order to facilitate my demonstration with a server, production environment is not recommended to do so (useless), just to record the process, as f ...

Posted by dtyson2000 on Mon, 12 Aug 2019 12:20:54 +0200

springboot project front-end and back-end separately use shiro to solve different sessionid problems

Problems encountered When developing app s or projects with separate front and back ends, sessions are different each time when using ajax and other ways to access the background, so using Shiro verification is totally impossible to pass. Originally, we used the way of token, which can realize the pro ...

Posted by showman on Mon, 12 Aug 2019 07:43:04 +0200