1. Overview

1. Problems faced by distributed system

In the complex distributed architecture system, the failure of mutual service calls cannot be avoided. If the call link of A service is relatively long, the probability of such call failure will be higher. For example, if A calls B, B calls C, C calls D, or A service calls need to rely on multiple services to cooperate, the probability of such call failure will also be higher. For example, A needs to call B, C, D services. Only B, C, D services are completed, and A service is finished Business can be done.

Fan out: the number of subordinate modules directly called by the current module.

If the response time of a service call on the fan-out link is too long or unavailable, the call to the current module will occupy more and more resources, and even cause system crash. This is the so-called system avalanche effect.

For a high traffic application, a single back-end dependency may cause all resources on all servers to saturate in a very short time. What's worse, the application may also cause the increase of delay between services, the shortage of resources such as queues and threads, and further more failures of the whole system.

In order to avoid such a situation, it is necessary to isolate the fault and delay, so as to avoid the normal operation of other services due to the problems of one service.

When a module has a problem, if the module is still accepting the request and calls other modules, it will lead to cascading failure and system avalanche effect.

2. What is hystrix

Hystrix is an open source library for dealing with delay and fault tolerance of distributed systems. In distributed systems, many dependencies inevitably lead to call failure, common timeout, program exception error, etc. hystrix can ensure that in the case of a dependency failure, it will not lead to overall service failure, avoid cascading failures, and thus improve the flexibility of distributed systems.

The circuit breaker itself is a kind of switch device. When a service fails, it will return an expected and processable alternative response (FallBack) to the caller through the fault monitoring of the circuit breaker (similar to fuse blowing), instead of waiting for a long time or throwing an exception that the caller cannot handle, which guarantees the service caller's Threads will not be occupied for a long time and unnecessarily, so as to avoid the spread of faults in the distributed system, or even avalanche.

3. What can hystrix do

Service degradation, service fusing and near real-time monitoring.

4. Official website of hystrix

https://github.com/Netflix/Hystrix/wiki/How-To-Use

from https://github.com/Netflix/Hystrix It can be seen from the information of Hystrix Status that hystrix is no longer under development. From the update of github, it has not been updated for a long time. Hystrix is recommended Resilience4j To replace, however, there are fewer large factories using Resilience4j in China. More large factories use Spring Cloud Alibaba Sentinel to realize fusing and current limiting, or to package on the existing open-source framework and realize their own functions.

Although Hystrix is no longer updated, the design concept of Hystrix is worth learning. It also paves the way for learning Spring Cloud Alibaba Sentinel later.

2. Important concepts of hystrix

1. Service degradation (fallback)

In case of exception, timeout, service fuse, thread pool and full semaphore in the program, service degradation will occur. At this time, the service will return a friendly prompt. Please try again later to prevent the client from waiting and returning a good prompt immediately.

2. Service break

When the number of service requests reaches a certain limit, service fusing can occur. Similar to the fuse at home, when the current is high, it will trigger fusing. At this time, the service is directly inaccessible, and the subsequent service degradation can be used to immediately return a response to the client.

3. Service flow limit

High concurrency requests, such as the seckill stage, are often used at a certain time. In order not to kill the service, some requests will be limited, such as put into the queue and let it wait for a while.

3.Hystrix case

1. Build the project

First, change cloud Eureka server 7001 and cloud Eureka server 7002 to stand-alone version. Later, the startup will be faster. Change the value of defaultZone to point to the address of your own service instead of registering with each other.

New cloud provider hystrix payment8001 module, modify pom.xml , add spring cloud starter Netflix hystrix dependency here.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.example</groupId>
    <artifactId>cloud-provider-hystrix-payment8001</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>com.atguigu.springcloud</groupId>
            <artifactId>cloud-api-commons</artifactId>
            <version>${project.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

add to application.yml Profile.

server:
  port: 8001
spring:
  application:
    name: cloud-provider-hystrix-payment
eureka:
  client:
    register-with-eureka: true
    fetch-registry: true
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka/

Create the main startup class.

package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@SpringBootApplication
@EnableEurekaClient
public class PaymentHystrixMain8001 {
    public static void main(String[] args) {
        SpringApplication.run(PaymentHystrixMain8001.class, args);
    }
}

Create business classes (Service and Controller).

package com.atguigu.springcloud.service;

import org.springframework.stereotype.Service;

@Service
public class PaymentService {
    public String paymentInfo_OK(Integer id) {
        return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_OK, id=" + id;
    }

    public String paymentInfo_TimeOut(Integer id) throws InterruptedException {
        int time = 3000;
        Thread.sleep(time);
        return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOut, id=" + id + "\t Time consuming:" + time;
    }
}

package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
@Slf4j
public class PaymentController {
    @Resource
    private PaymentService paymentService;
    @Value("${server.port}")
    private String serverPort;

    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        String result = paymentService.paymentInfo_OK(id);
        log.info("result={}", result);
        return result;
    }

    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
        String result = paymentService.paymentInfo_TimeOut(id);
        log.info("result={}", result);
        return result;
    }
}

First start the cloud Eureka server 7001 module, then start the cloud provider hystrix payment 8001 module, and access through the browser http://localhost:8001/payment/hystrix/ok/1 and http://localhost:8001/payment/hystrix/timeout/1 Check the effect. At this time, both requests can be accessed normally, but the timeout request is slower. Next, we will expand based on these two requests to demonstrate the functions in Hystrix.

2. High concurrency test

Using high concurrency testing tools Apache Jmeter Test.

Right click the test plan → add → thread (user) → thread group, and add a thread group named Spring Cloud. The number of threads is 200, the ramp up time is 1, and the number of cycles is 100. The rest parameters can be saved by default. Right click on Spring Cloud thread group → add → sampler → HTTP request, name cloud provider hystrix payment8001, server name or IP is localhost, port number is 8001, HTTP request is GET request, path is http://localhost:8001/payment/hystrix/timeout/1 , click save, and try the timeout request first. Click the green arrow in the menu bar to initiate the request. At this time, you can access it through the browser http://localhost:8001/payment/hystrix/ok/1 You'll find that it's slowed down.

New cloud consumer feign hystrix order 80 module, modify pom.xml .

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>cloud-consumer-feign-hystrix-order80</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

add to application.yml Profile.

server:
  port: 80
# Here, feign is only used as the client, not registered in eureka
eureka:
  client:
    # Indicates whether to register yourself into EurekaServer, which is true by default
    register-with-eureka: false
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka/
spring:
  application:
    name: cloud-consumer-feign-hystrix-order

Add the main startup class.

package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.openfeign.EnableFeignClients;

@SpringBootApplication
@EnableFeignClients
public class OrderHystrixMain80 {
    public static void main(String[] args) {
        SpringApplication.run(OrderHystrixMain80.class, args);
    }
}

Add business classes (Service and Controller).

package com.atguigu.springcloud.service;

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")
public interface PaymentHystrixService {
    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id);

    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException;
}

package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentHystrixService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
public class OrderHystrixController {
    @Resource
    PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        return paymentHystrixService.paymentInfo_OK(id);
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
        return paymentHystrixService.paymentInfo_TimeOut(id);
    }
}

Start cloud Eureka server 7001, cloud provider hystrix payment 8001 and cloud consumer feign hystrix order 80 successively.

Browser access http://localhost:8001/payment/hystrix/ok/1,http://localhost:8001/payment/hystrix/timeout/1,http://localhost/consumer/payment/hystrix/ok/1 Can be accessed normally when http://localhost/consumer/payment/hystrix/timeout/1 The prompt is timeout. This is because of Feign. The default connection timeout of Feign is 1 second, and the default read resource timeout is 1 second. This does not matter for the moment. Of course, it can also be based on the previous article OpenFeign Note, modification timeout.

At this time, start Apache JMeter to http://localhost:8001/payment/hystrix/timeout/1 Request to send high concurrent access through browser http://localhost/consumer/payment/hystrix/ok/1 , it will be found that there is a direct timeout. At this time, the simulation situation is: a large number of requests hit the producer. At this time, another external request through Feign will appear timeout. When the pressure test is not run, this request http://localhost/consumer/payment/hystrix/ok/1 The response time is still very fast.

Reason: the 8001 service interface is occupied by a large number of requests, and the processing speed cannot keep up with it. The Tomcat worker thread is full, and the later arriving thread can only wait, which leads to the browser accessing the request that should be returned immediately http://localhost/consumer/payment/hystrix/ok/1 There was a timeout.

It is because of this kind of fault or poor situation that we need degradation, fault tolerance, current limiting and other technologies to deal with this problem.

• Producer 8001 timeout, consumer 80 can't wait all the time, there must be service degradation
• Producer 8001 is down, consumer 80 can't wait all the time, service must be degraded
• The producer 8001 is normal, the consumer 80 has its own fault or the required waiting time is less than the processing time of the producer, and the consumer 80 handles the degradation of the service by itself

3. Service degradation

First, transform from the service producer to find the paymentInfo_TimeOut() method, because the execution of this method is a little long, we add a standard to it. When the execution of the method fails to meet the standard requirements, the target method will fail quickly, and then the method of service degradation will be executed, instead of waiting for the return of the target method.

package com.atguigu.springcloud.service;

import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;

@Service
public class PaymentService {
    public String paymentInfo_OK(Integer id) {
        return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_OK, id=" + id;
    }

    // @The HystrixCommand annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method will be terminated and then the method specified by fallbackMethod will be executed
    @HystrixCommand(fallbackMethod = "paymentInfo_TimeOutHandler", commandProperties = {
            @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")
    })
    public String paymentInfo_TimeOut(Integer id) throws InterruptedException {
        int time = 5000;
        // int a = 1 / 0; / / when the program reports an error, the service degradation method will also be triggered
        Thread.sleep(time);
        return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOut, id=" + id + "\t Time consuming:" + time;
    }

    // Method of service degradation
    // It should be noted that the service degradation method should be consistent with the method signature of the target method, that is, the parameters and return values should be consistent, otherwise, it will be prompted that the service degradation method cannot be found
    public String paymentInfo_TimeOutHandler(Integer id) {
        return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOutHandler, id=" + id + "\t Service degradation method is running";
    }
}

Add the @ enablechircuitbreaker annotation on the main startup class to enable the breaker function, launch EurekaMain7001 and PaymentHystrixMain8001 modules, and access the browser http://localhost:8001/payment/hystrix/timeout/1 During the test, the method body business takes 5 seconds, and the caller can wait for 3 seconds at most. When the time is more than 3 seconds, and it has not returned, the paymentinfo is executed directly_ Timeouthandler () method, on the browser side, we can see the specific output.

Above, we have dealt with the service degradation of the service producer. Below, we have dealt with the service consumer. We need to pay attention to that: service degradation can be placed on both the producer and the consumer, more often on the consumer side.

Add the @ EnableHystrix annotation to the main boot class. Business class, imitating the description of service producer, makes the same modification.

package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
public class OrderHystrixController {
    @Resource
    PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        return paymentHystrixService.paymentInfo_OK(id);
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    // @The HystrixCommand annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method will be terminated, and then the method specified by fallbackMethod will be executed
    @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMehtod", commandProperties = {
            @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "2500")
    })
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
        // int a = 1 / 0; / / when the program reports an error, the service degradation method will also be triggered
        return paymentHystrixService.paymentInfo_TimeOut(id);
    }

    // Method of service degradation
    // It should be noted that the service degradation method should be consistent with the method signature of the target method, that is, the parameters and return values should be consistent, otherwise, it will be prompted that the service degradation method cannot be found
    public String paymentTimeOutFallbackMehtod(@PathVariable("id") Integer id) {
        return "Consumer call producer busy, please wait and try again";
    }
}

At this time, the service running time of the timeout() method on the service producer side is 5S. After the service degradation policy is added, if the timeout() method can't be completed within 3s, the service degradation method will be executed. The requirements of the service consumer side are more stringent. The consumer can wait for 2.5s at most, and if it can't be returned within 2.5s, the degradation method will be executed. Access through browser http://localhost/consumer/payment/hystrix/timeout/1 , observed in the Network, it was found that more than 2 seconds triggered the service degradation method. This is because we did not set the default timeout of Ribbon (1s for connection establishment and 1s for reading resources) to 2s, which is less than 2.5 seconds set by HystrixProperty. First, the timeout of Ribbon was triggered, the code reported an error, and then the degradation method was triggered. We modify application.yml Profile, modify the Ribbon timeout.

# Set the feign client timeout. OpenFeign supports Ribbon by default
ribbon:
  ReadTimeout: 5000 # Time spent reading resources after connection established
  ConnectTimeout: 5000 # Time taken to establish connection

At this time, send again http://localhost/consumer/payment/hystrix/timeout/1 For the request, check the time through the Network. It is found that the return time is more than 2.5 seconds. At this time, the 2.5 seconds set by HystrixProperty is working. In addition, the information output by the browser is the output of the degradation method in the client.

To demonstrate another situation, change the time of the client's HystrixProperty to 3.5s, which will not be set in general, because the client's 3.5s is greater than the server's 3s. Here, we only do a test and use it and access it through the browser http://localhost/consumer/payment/hystrix/timeout/1 , observe the Network time, 3s will return, and the content displayed by the browser is the output of the server degradation method.

The timeout in this place is a bit disorderly. I think it can be understood. Look at these timeout times. The minimum timeout is the standard. As long as the minimum timeout is reached, service degradation will be caused. The cause of service degradation is determined by the minimum timeout.

When the service degradation is added to the producer, the service producer is called directly, which is triggered by 3s of HystrixProperty in the producer Controller.

When the service degradation is added to the consumer, directly call the service consumer:

• If the default timeout of the Ribbon has not been modified, and the consumer calls more than the default 2s, at this time, the trigger for service degradation is min (2s of the Ribbon, the time specified by the consumer's HystrixProperty). The smaller the value is, the service degradation caused by the smaller the value is.
• If the default timeout of the Ribbon has been modified, and the modified value is assumed to be very large, excluding the service degradation caused by the timeout of the Ribbon call, then the trigger condition for service degradation is the time specified by the consumer's HystrixProperty. If the service producer also specifies the time in the HystrixProperty, then in min (the time specified by the consumer's HystrixProperty, the producer's hystrix Xproperty) as the degradation time, which value is small is the degradation triggered by which, and which degradation method to take.

Look back at the configured service degradation methods. If there are 10 methods that need service degradation, you need to configure 10 additional service degradation methods. Is there any way to use a common method to deal with service degradation, and then do custom configuration for the rest? Service degradation method and business method are put together, which leads to chaos in business class. Can we take the degradation method out?

Add a global service degradation method, add @ DefaultProperties(defaultFallback = "paymentGlobalFallbackMethod" on the class), indicating all methods with @ HystrixCommand in the class. If there is no special indication on the method, the paymentGlobalFallbackMethod() method will be used when the method is downgraded.

package com.atguigu.springcloud.controller;

import com.atguigu.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
// Specifies a global degradation method that works for methods annotated with @ HystrixCommand
@DefaultProperties(defaultFallback = "paymentGlobalFallbackMethod")
public class OrderHystrixController {
    @Resource
    PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id) {
        return paymentHystrixService.paymentInfo_OK(id);
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
    // @The HystrixCommand annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method will be terminated, and then the method specified by fallbackMethod will be executed
    // @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMehtod", commandProperties = {
    //         @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3500")
    // })
    // With the @ HystrixCommand annotation, it means that if you follow the service degradation policy of Hystrix, and you do not specify a custom callback at this time, you will follow the default callback
    @HystrixCommand(commandProperties = {@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "2500")})
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
        // int a = 1 / 0; / / when the program reports an error, the service degradation method will also be triggered
        return paymentHystrixService.paymentInfo_TimeOut(id);
    }

    // Method of service degradation
    // It should be noted that the service degradation method should be consistent with the method signature of the target method, that is, the parameters and return values should be consistent, otherwise, it will be prompted that the service degradation method cannot be found
    public String paymentTimeOutFallbackMehtod(@PathVariable("id") Integer id) {
        return "Consumer call producer busy, please wait and try again";
    }

    // Global service degradation method, @ HystrixCommand, if fallbackMethod is not specified, global service degradation method will be used
    public String paymentGlobalFallbackMethod() {
        return "Global service degradation method, please try again later";
    }
}

Browser access http://localhost/consumer/payment/hystrix/timeout/1 , the service can still be demoted according to the timeout rule specified by us. When the service is demoted, the method to execute is paymentGlobalFallbackMethod().

To achieve decoupling, you can use the fallback parameter in the @ FeignClient annotation. New pa ymentFallbackService.java To implement the PaymentHystrixService interface and rewrite the methods in the interface, the two methods rewritten here are the methods used for service degradation execution. Finally, write the PaymentFallbackService class name to the fallback parameter in the @ FeignClient annotation to achieve the decoupling effect between the business method and the degradation method.

package com.atguigu.springcloud.service;

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT", fallback = PaymentFallbackService.class)
public interface PaymentHystrixService {
    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id);

    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException;
}

package com.atguigu.springcloud.service;

import org.springframework.stereotype.Component;

/**
 * When the call in PaymentHystrixService goes wrong, the corresponding method here will be executed to decouple the business method and the degradation method
 */
@Component
public class PaymentFallbackService implements PaymentHystrixService {
    @Override
    public String paymentInfo_OK(Integer id) {
        return "paymentInfo_OK()Method corresponding degradation method";
    }

    @Override
    public String paymentInfo_TimeOut(Integer id) throws InterruptedException {
        return "paymentInfo_TimeOut()Method corresponding degradation method";
    }
}

modify application.yml , enable Hystrix. The default value is false. This place seems to conflict with the previous timeout configuration. If Hystrix is enabled, which timeout does not work and why is unknown.

feign:
  hystrix:
    enabled: true

Access through browser first http://localhost/consumer/payment/hystrix/ok/1 , the value returned by the producer can be obtained through the service producer. At this time, the producer can be shut down to simulate the downtime of the producer and access again http://localhost/consumer/payment/hystrix/ok/1 , you will see that it executes the paymentinfo in the PaymentFallbackService_ OK () method, now the service degradation method and business method are separated.

4. Service fuse

Fusing mechanism is a protection mechanism for medium and micro service links to deal with avalanche effect. When a micro service with fan out filing rate makes an error or the response time is too long, it will degrade the service, and then fusing the call of the micro service of the node to quickly return the wrong response information. When it is detected that the microservice call response of this node is normal, the call link is restored.

In the Spring Cloud framework, the fusing mechanism is implemented by Hystrix, which monitors the calls between microservices. When the failed calls reach a certain threshold, the fusing mechanism will be triggered (default is 20 failures in 5 seconds). The annotation of fusing mechanism is @ HystrixCommand.

Service fuse related content expansion: https://martinfowler.com/bliki/CircuitBreaker.html.

Add methods to the PaymentService of cloud provider hystrix payment8001 to demonstrate service fusing.

// The service is broken. The HystrixProperty configured here can be found in the https://github.com/Netflix/Hystrix/wiki/Configuration View, or view the HystrixCommandProperties class
@HystrixCommand(fallbackMethod = "paymentCircuitBreakerFallback", commandProperties = {
        @HystrixProperty(name = "circuitBreaker.enabled", value = "true"),// Whether to open the circuit breaker
        // The number of requests in a rolling window. When the number of requests reaches 10, the failure rate is calculated, so as to judge whether the condition of open circuit is met
        @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"),
        // After triggering the open circuit, it will directly fail within 10s. After more than 10s, try to recover once
        @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"),
        // Open circuit when failure rate reaches 60%
        @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "60")
})
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
    if (id < 0) {
        throw new RuntimeException("id Cannot be negative");
    }
    String uuid = UUID.randomUUID().toString();
    return Thread.currentThread().getName() + "\t Call successful, UUID=" + uuid;
}
// Service degradation method
public String paymentCircuitBreakerFallback(@PathVariable("id") Integer id) {
    return "id It can't be negative. Please try again later, id=" + id;
}

With the Service, there must be a Controller to implement the call. Add a method to the PaymentController of cloud provider hystrix payment8001.

@GetMapping("/payment/circuit/{id}")
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
    String result = paymentService.paymentCircuitBreaker(id);
    log.info("result={}", result);
    return result;
}

Start eurekmain7001 and PaymentHystrixMain8001 modules. Browser send http://localhost:8001/payment/circuit/1 Request, can get the correct return value, try to send a large number of http://localhost:8001/payment/circuit/-1 Request, send one more http://localhost:8001/payment/circuit/1 Request: for this normal request, the service degradation method is still used, which means that the circuit breaker is disconnected at this time and has not been recovered. After sending several correct requests again, the fuse recovers and can handle the normal request.

In case of fusing, the fusing time will be recorded. According to the configuration here, in the next 10 seconds, the request will go through the method of service degradation. After 10 seconds, try to call the service. If it can be turned on, recover the calling link. If it can't be turned on, continue to keep fusing and record the time again.

There are three types of fusing: fusing open, fusing half open and fusing close.

• Fusing on: Hystrix is fusing, the new request is not calling the service, but directly running the service degradation method, recording the fusing time. When the fusing duration reaches the set sleepWindowInMilliseconds, it will enter fusing half open
• Fuse half open: some requests for service call. If the call succeeds, fuse is closed. If the call fails, keep the fuse open state and record the time again
• Fuse off: normal service call

5. Service current limitation

It is described in Sentinel of subsequent Alibaba.

4.Hystrix workflow

Go to the Hystrix page and click How it Works.

Here's a small problem. If the picture on GitHub doesn't show, you can try this method to solve it.

visit https://www.ipaddress.com/ , post the domain name showing the picture, here my is raw.Githubusercontent.com , will get an IP address. Write the IP address and domain name to the local hosts, refresh the page again, and the picture will be displayed. The reason is like DNS pollution or DNS cache.

If you still can't visit my friends, please look at this picture first. I'll post it. The whole process consists of 9 steps:

1. Create a HystrixCommand or hystrixobservercommand object
2. Execute Command command
3. Check whether there is available cache. If so, return the contents of cache directly. If not, go to step 4
4. Check the opening and closing of the fuse. If it is opened, run the service degradation method. Step 8. If it is not opened, step 5
5. Check whether the thread pool, queue and semaphore are sufficient. If not, run the service degradation method. Step 8. If sufficient, step 6
6. Execute the construct() method of the HystrixObservableCommand or the run() method of the HystrixObservableCommand
7. According to the execution results, judge whether the execution is successful or not and whether there is timeout, and feed back to the health judgment function in the circuit, thus affecting the opening and closing of the circuit breaker
8. In case of disconnection of fuse, insufficient resources, execution failure, execution timeout, etc., go to the service degradation method to obtain the return value
9. Everything is normal. The circuit breaker is closed. Call the service normally to get the return value

5. Service monitoring Hystrix Dashboard

In addition to isolating the calls of dependent services, hystrix also provides a quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated by hystrix, and display it to users in the form of statistical reports and graphs, including how many successful requests are executed each second and how many failures are. Netflix has realized the monitoring of the above indicators through the project of hystrix metrics event stream. Spring Cloud also provides the integration of the Hystrix Dashboard to transform monitoring content into a visual interface.

New cloud consumer hystrix dashboard9001 module, modify pom.xml , the spring cloud starter Netflix hystrix dashboard coordinates are introduced.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

add to application.yml Configuration file indicating the port number.

server:
  port: 9001

Add the main startup class, add the @ EnableHystrixDashboard and @ enablechircuitbreaker annotations, and add a bean to specify the monitoring path. Otherwise, the content cannot be monitored.

package com.atguigu.springcloud;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;

@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardMain9001 {
    public static void main(String[] args) {
        SpringApplication.run(HystrixDashboardMain9001.class, args);
    }

}

To achieve the monitoring function, you need to pom.xml The coordinates of the spring boot starter actor are in. Start the HystrixDashboardMain9001 module, and access the browser http://localhost:9001/hystrix You can see the interface.

In order to demonstrate the monitoring effect, that is, the 9001 module monitors the operation of the 8001 module, modify the cloud provider hystrix payment8001 module, and check the 8001 module pom.xml If there is no spring boot starter actor coordinate, it must be.

Modify the main startup class of PaymentHystrixMain8001 and inject a Bean to specify the monitoring path. Otherwise, an error of Unable to connect Command Metric Stream will be reported. Note that the main startup class of 8001 is modified.

package com.atguigu.springcloud;

import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
@EnableEurekaClient
@EnableCircuitBreaker
public class PaymentHystrixMain8001 {
    public static void main(String[] args) {
        SpringApplication.run(PaymentHystrixMain8001.class, args);
    }

    /**
     * This configuration is for service monitoring, and has nothing to do with service fault tolerance itself. It is a pit after Spring Cloud upgrade
     * Because the default path of ServletRegistrationBean in SpringBoot is not "/ hystrix.stream "
     * So you need to configure the servlet in your own project
     */
    @Bean
    public ServletRegistrationBean getServlet() {
        HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet() ;
        ServletRegistrationBean registrationBean = new ServletRegistrationBean(streamServlet);
        registrationBean.setLoadOnStartup(1);
        registrationBean.addUrlMappings("/hystrix.stream");
        registrationBean.setName("HystrixMetricsStreamServlet");
        return  registrationBean;
    }
}

After the modification, start Eureka7001 registration center, Provider8001 service producer, Dashboard9001 service monitoring, and then click http://localhost:9001/hystrix In the input box of the page, enter http://localhost:8001/hystrix.stream , that is to say, which service to monitor? The default value of Delay is 2000, and the Title can be any name. Click "Monitor Stream" to start monitoring, open a new window and visit it for many times http://localhost:8001/payment/circuit/1 , under the Circuit label, you can see a broken line and a dot. The broken line is to record the relative change of the access flow in the last two minutes. When the access flow is large, the broken line will rise and the dot will become larger. At the same time, the dot will change color according to the health status, and the health degree will decrease from green to yellow to orange to red in turn. There is also a Circuit, indicating whether it is fused, If it is Closed, it means it is not fused; if it is Open, it means it is fused.

In the upper right corner of the monitoring page, there are 7 colors, corresponding from left to right: the number of successful requests, the number of request fusing, the number of error requests, the number of timeout requests, the number of requests rejected by the thread pool, the number of failed requests, and the error rate in the last 10 seconds.

For more information about Dashboard, please refer to: https://github.com/Netflix-Skunkworks/hystrix-dashboard/wiki