1, Core knowledge of service fault tolerance

1.1 avalanche effect

In microservice architecture, it is very common for A request to call multiple services. For example, when the client accesses service A, service A needs to call service B, and service B needs to call service C. due to network reasons or its own reasons, if service B or service C cannot respond in time, service A will be blocked until service B and Service C respond. At this time, if A large number of requests flood in, the thread resources of the container will be consumed, resulting in service paralysis. The dependence between services will spread and cause chain reaction, which will have disastrous and serious consequences for the whole microservice system. This is the "avalanche" effect of service failure.

Avalanche is the butterfly effect in the system, which is caused by various reasons, including unreasonable capacity design, slow response of a method under high concurrency, or depletion of resources of a machine. From the source, we can not completely eliminate the occurrence of avalanche source, but the root cause of avalanche comes from the strong dependence between services, so we can evaluate in advance and do a good job in fusing, isolation and current limitation.

1.2 service isolation

As the name suggests, it means that the system is divided into several service modules according to certain principles, and each module is relatively independent without strong dependence. When a fault occurs, it can isolate the problem and impact within a module without spreading the risk, affecting other modules and affecting the overall system service.

1.3 fuse degradation

The concept of fusing comes from the Circuit Breaker in electronic engineering. In the Internet system, when the downstream service responds slowly or fails due to excessive access pressure, the upstream service can temporarily cut off the call to the downstream service in order to protect the overall availability of the system. This measure of sacrificing the part and preserving the whole is called fusing.

The so-called degradation means that when a service is blown, the server will no longer be called. At this time, the client can prepare a local fallback callback and return a default value. It can also be understood as the bottom method.

1.4 service current limit

Current limiting can be considered as a kind of service degradation. Current limiting is to limit the input and output flow of the system to achieve the purpose of protecting the system. Generally speaking, the throughput of the system can be measured. In order to ensure the stable operation of the system, once the threshold that needs to be limited is reached, it is necessary to limit the flow and take a few measures to complete the purpose of limiting the flow. For example: postpone the solution, refuse to solve, or partially refuse to solve, etc.

2, Introduction to Hystrix

Hystrix is a delay and fault tolerance library open source by Netflix. It is used to isolate access to remote systems, services or third-party libraries to prevent cascading failures, so as to improve the availability and fault tolerance of the system. Hystrix mainly realizes delay and fault tolerance through the following points.

• Wrap request: use HystrixCommand to wrap the calling logic of the dependency. Each command is executed in a separate thread. This use
• The command mode in design mode is.
• Trip mechanism: when the error rate of a service exceeds a certain threshold, Hystrix can automatically or manually trip and stop requesting the service for a period of time.
• Resource isolation: Hystrix maintains a small thread pool (or semaphore) for each dependency. If the thread pool is full, the request sent to the dependency will be rejected immediately instead of waiting in line, so as to speed up the failure determination.
• Monitoring: Hystrix can monitor the changes of operation indicators and configurations in near real time, such as success, failure, timeout, and rejected requests.
• Fallback mechanism: execute fallback logic when the request fails, times out, is rejected, or when the circuit breaker is open. The fallback logic is provided by the developer, for example, returning a default value.
• Self repair: after the circuit breaker is opened for a period of time, it will automatically enter the "half open" state.

3, Rest implementation service

3.1 configuration dependency

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>

3.2 open fuse

Add the @ enablercircuitbreaker annotation in the startup class OrderApplication to open the support for fuses.

//@Enablercircuitbreaker / / turn on the fuse
//@SpringBootApplication
@SpringCloudApplication
public class OrderApplication {
//Create RestTemplate object
@Bean
@LoadBalanced
public RestTemplate restTemplate() {
return new RestTemplate();
}
public static void main(String[] args) {
SpringApplication.run(OrderApplication.class, args);
}
}

We can see that there are more and more annotations on our classes. The above three annotations are often introduced into micro services, so Spring provides a composite annotation: @ SpringCloudApplication

3.3 configure fuse degradation business logic

@RestController
@RequestMapping("/order")
public class OrderController {
@Autowired
private RestTemplate restTemplate;
//Place an order
@GetMapping("/product/{id}")
@HystrixCommand(fallbackMethod = "orderFallBack")
public Product findProduct(@PathVariable Long id) {
return restTemplate.getForObject("http://shop-serviceproduct/product/1", Product.class);
}
//Degradation method
public Product orderFallBack(Long id) {
Product product = new Product();
product.setId(-1l);
product.setProductName("Fuse:Trigger degradation method");
return product;
}
}

It can be seen from the code that a fallback method findProductFallBack is written for the findProduct method. This method has the same parameters and return value type as the findProduct method. This method returns a default error message.

On the Product method, use the fallbackMethod attribute of the annotation @ HystrixCommand to specify that the degradation method triggered by fusing is findProductFallBack.

• Because the degraded logic method of fuse must guarantee the same parameter list and return value declaration as the normal logic method.
• On the findProduct method, HystrixCommand(fallbackMethod = "findProductFallBack") is used to
  Declare a method of degrading logic

3.3.1 default Fallback

We just wrote fallback on a business method. If there are many such methods, don't we have to write a lot. Therefore, we can add the fallback configuration to the class to realize the default fallback:

3.3.2 timeout setting

In the previous case, the request will return an error message after more than 1 second. This is because the default timeout of Hystix is 1. We can modify this value through configuration:

hystrix:
	command:
		default:
			execution:
				isolation:
					thread:
						timeoutInMilliseconds: 2000

4, Feign implementation service

Spring cloud fegin integrates hystrix for Feign by default, so you don't need to add hystrix after adding Feign dependencies. So how can Feign's circuit breaker mechanism take effect? Just follow the following steps:

4.1 modify application YML enable hystrix in Fegin

Hystrix has been built in Feign, but it is closed by default. It needs to be in the application of the project Enable support for hystrix in YML

feign:
	hystrix: #Turn on the hystrix fuse in feign
		enabled: true

4.2 configure the implementation class of FeignClient interface

To implement fuse degradation based on Feign, the degradation method needs to be configured in the implementation class of Feign client interface

/**
* Implement the customized ProductFeginClient interface
* Write a method in the interface implementation class
*/
@Component
public class ProductFeginClientCallBack implements ProductFeginClient {

	/**
	* Degradation method
	*/
	public Product findById(Long id) {
		Product product = new Product();
		product.setId(-1l);
		product.setProductName("Fuse:Trigger degradation method");
		return product;
	}
}

4.3 modify FeignClient and add hystrix

Add degradation method in @ FeignClient annotation

//Specify the name of the micro service to be called
@FeignClient(name="shop-service-product",fallback =
ProductFeginClientCallBack.class)
public interface ProductFeginClient {
	//Request path to call
	@RequestMapping(value = "/product/{id}",method = RequestMethod.GET)
	public Product findById(@PathVariable("id") Long id);
}

@FeignClient annotation declares the degradation method with fallback