Spring cloud integrated Bus message Bus

Posted by jcran on Wed, 08 Dec 2021 09:14:38 +0100

SpringCloud

Bus message bus

What is the Bus message Bus

In a word, the distributed automatic refresh configuration function.

What is it?

Spring Cloud Bus can be used with Spring Cloud Config to dynamically refresh the configuration.

Spring Cloud Bus is a framework used to link the nodes of distributed systems with lightweight messaging systems. It integrates the event processing mechanism of Java and the functions of message middleware.

Spring cloud bus currently supports RabbitMQ and Kafka.

What can I do

Spring Cloud Bus can manage and propagate messages between distributed systems, just like a distributed actuator. It can be used to broadcast state changes, push events, etc. it can also be used as a communication channel between microservices.

Why is it called a bus

What is a bus

In microservice architecture systems, lightweight message brokers are usually used to build a common message topic and connect all microservice instances in the system. Since the messages generated in this topic will be monitored and consumed by all instances, it is called message bus. Each instance on the bus can easily broadcast some messages that need to be known by other instances connected to the subject.

Basic principles

ConfigClient instances listen to the same Topic in MQ (spring cloud bus by default). When a service refreshes the data, it will put this information into the Topic, so that other services listening to the same Topic can be notified, and then update their own configuration.

RabbitMQ environment configuration of Bus

Installing MQ under Linux

linux Installation of MQ

Installing MQ under windows

  • Install Erlang, download address: http://erlang.org/download/otp_win64_21.3.exe
  • Install RabbitMQ at: https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.8.3/rabbitmq-server-3.8.3.exe
  • Open cmd and enter the SBIN directory under the RabbitMQ installation directory, such as D:\devSoft\RabbitMQ Scrverk\rabbitmq_server-3.7.14\sbin
  • Enter the following command to start the management function
  • rabbitmq-plugins enable rabbitmq _management
  • This allows you to add a visualization plug-in.
  • Visit the address to see if the installation was successful: http://localhost:15672/
  • Enter the account and password and log in: guest guest

Design idea and selection of Bus dynamic refresh global broadcast

You must have a good RabbitMQ environment first

Demonstrate the broadcast effect, increase the complexity, and then make another 3366 with 3355 as the template

1. Create cloud-config-client-3366

2.POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud_Parent</artifactId>
        <groupId>dhy.xpy</groupId>
        <version>520.521.finally</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-config-client-3366</artifactId>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencies>
        <!--Add message bus RabbitMQ support-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bus-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

3.YML

This is bootstrap.yml

server:
  port: 3366

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name and the above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address

  #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port for MQ access
  rabbitmq:
    host: localhost
    port: 5672
    username: admin
    password: 123

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

4. Main startup

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@EnableEurekaClient
@SpringBootApplication
public class ConfigClientMain3366
{
    public static void main(String[] args)
    {
        SpringApplication.run(ConfigClientMain3366.class,args);
    }
}

5.controller

@RestController
@RefreshScope
public class ConfigClientController
{
    @Value("${server.port}")
    private String serverPort;

    @Value("${config.info}")
    private String configInfo;

    @GetMapping("/configInfo")
    public String configInfo()
    {
        return "serverPort: "+serverPort+"\t\n\n configInfo: "+configInfo;
    }

}

design idea

1. Use the message bus to trigger a client / bus/refresh and refresh the configuration of all clients

2. Use the message bus to trigger the / bus/refresh endpoint of a server ConfigServer and refresh the configuration of all clients

The architecture in Figure 2 is obviously more suitable. The reasons for the inapplicability are as follows:

  • It breaks the single responsibility of microservices, because microservices are business modules, which should not bear the responsibility of configuration refresh.
  • It destroys the peer-to-peer of microservice nodes.
  • There are certain limitations. For example, when a microservice is migrated, its network address often changes. At this time, if you want to refresh automatically, you will add more modifications.

Implementation of Bus dynamic refresh global broadcast configuration

Add message bus support to cloud-config-center-3344 configuration center server

POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3344

spring:
  application:
    name:  cloud-config-center #Microservice name registered into Eureka server
  cloud:
    config:
      server:
        git:
          uri: https://gitee.com/DaHuYuXiXi/springcloud-config.git #GitHub the GIT warehouse name above
          ####search for directory
          search-paths:
            - springcloud-config
      ####Read branch
      label: master
#rabbitmq related configuration
  rabbitmq:
   host: 192.168.112.128
   port: 5672
   username: admin
   password: 123

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

##rabbitmq related configuration, exposing the endpoint of bus refresh configuration
management:
  endpoints: #Exposed bus refresh configured endpoint
    web:
      exposure:
        include: 'bus-refresh'

Add message bus support to cloud-config-client-3355 client

POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3355

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name and the above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address k

  #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port for MQ access
  rabbitmq:
    host: 192.168.112.128
    port: 5672
    username: admin
    password: 123

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

Add message bus support to cloud-config-client-3366 client POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3366

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name and the above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address

  #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port for MQ access
  rabbitmq:
    host: 192.168.112.128
    port: 5672
    username: admin
    password: 123

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

test

start-up

  • EurekaMain7001
  • ConfigcenterMain3344
  • ConfigclientMain3355
  • ConfigclicntMain3366

Operation and maintenance engineer

  • Modify the content of the configuration file on Github and increase the version number
  • Send POST request 
curl -X POST "http://localhost:3344/actuator/bus-refresh"

Send once, take effect everywhere

Configuration center

http://localhost:3344/config-dev.yml

client

  • http://localhost:3355/configInfo
  • http://localhost:3366/configInfo
  • Get the configuration information and find that it has been refreshed

Bus dynamic refresh fixed point notification

I don't want to be notified in full, but I just want to be notified at a fixed point

  • Notice only 3355
  • No notice 3366

In a word - specify a specific instance to take effect instead of all

Formula: http://Localhost: 3344 / Actor / bus refresh / {destination} --- > microservice name + port number
/bus/refresh The request is no longer sent to the specific service instance, but to config server adopt destination The parameter class specifies the service or instance whose configuration needs to be updated

case

  • Here, we take refreshing the config client (application name set in the configuration file) running on port 3355 as an example. Only 3355 is notified, not 3366
curl -X POST "http://localhost:3344/actuator/bus-refresh/config-client:3355

Notification summary

Principle Exploration

spring cloud bus is integrated with spring cloud config, and RabbitMq is used as the message agent to realize the dynamic update of application configuration.

First, we sent a post request to each micro service separately to refresh the latest port information of the corresponding micro service and complete the manual refresh. Now we have two schemes to complete the broadcast refresh:

Send a post request to instance 3 of service A and access the / bus/refresh interface. At this time, instance 3 of service A will send a refresh request to the message bus. The message event will be obtained from the bus by instance 1 and instance 2 of service A, and their configuration information will be obtained from the config server again, so as to realize the dynamic update of configuration information.

1. Introduce spring cloud bus into config server and add the configuration server to the message bus;

2./bus/refresh requests are no longer sent to specific service instances, but to Config Server, and specify the services or instances that need to update the configuration through the destination parameter.

We have adopted the second scheme, so what is the principle of the second scheme?

Core process

Spring Cloud implements the function of dynamic refresh of the configuration center by default, which is in the Spring Cloud context package of the public module. At present, the popular configuration center Spring Cloud Config dynamic refresh depends on this module, and the Nacos dynamic refresh mechanism is extended on this module, which is more powerful and rich than Spring Cloud Config.

First, the dynamic refresh of Spring Cloud Config needs to rely on Spring Cloud Bus, while Nacos is directly pushed to various services after background modification.

Secondly, the refresh mechanism of Spring Cloud Config is for all modified variables. Only if there are changes, they will be obtained in the background.

Nacos supports finer granularity. Only configuration items with the refresh attribute of true will be changed to new values during operation. This is Nacos's unique way.

Similarities: the dynamic refresh ranges of the two configuration centers are as follows:

  • @ConfigurationProperties annotated configuration class
  • @RefreshScope annotated bean

The general core process is as follows:

Take a look at the implementation principles of these two points.

First, the spring cloud config dynamic refresh function is enabled through the following variables. The default value is true.

Obviously default spring cloud config The dynamic refresh function is enabled by default
@ConditionalOnProperty(value = "endpoints.refresh.enabled", matchIfMissing = true)

RefreshEndpoint endpoint exposure method:

@Configuration(
    proxyBeanMethods = false
)
@AutoConfigureAfter({WebMvcAutoConfiguration.class})
public class LifecycleMvcEndpointAutoConfiguration {
    public LifecycleMvcEndpointAutoConfiguration() {
    }

    @Bean
    @ConditionalOnMissingBean
    public EnvironmentManager environmentManager(ConfigurableEnvironment environment) {
        return new EnvironmentManager(environment);
    }
}

// Mvc adapter
public class GenericPostableMvcEndpoint extends EndpointMvcAdapter {

	//The proxy class is RefreshEndpoint 
	public GenericPostableMvcEndpoint(Endpoint<?> delegate) {
		super(delegate);
	}
    
    //When we access the refresh endpoint, we must send a post request
	@RequestMapping(method = RequestMethod.POST)
	@ResponseBody
	@Override
	public Object invoke() {
	     //If the RefreshEndpoint function is not enabled, the information that the endpoint is unavailable will be output to the page
	     //http status is not found
		if (!getDelegate().isEnabled()) {
			return new ResponseEntity<>(Collections.singletonMap(
					"message", "This endpoint is disabled"), HttpStatus.NOT_FOUND);
		}
		//Otherwise, call the invoke method of endpointmvcdadapter
		//In fact, the invoke method of the following RefreshEndpoint is called
		return super.invoke();
	}
}

The implementation method here is the same as the principle of springboot actor endpoint, which is implemented through the endpoint mvccadapter adapter.

RefreshEndpoint endpoint:

Version together:

public class RefreshEndpoint extends AbstractEndpoint<Collection<String>> {

	private ContextRefresher contextRefresher;

	public String[] refresh() {
		Set<String> keys = contextRefresher.refresh();
		return keys.toArray(new String[keys.size()]);
	}
	
	@Override
	public Collection<String> invoke() {
		return Arrays.asList(refresh());
	}
}

Current version:

@Endpoint(
    id = "refresh"
)
public class RefreshEndpoint {
    private ContextRefresher contextRefresher;

    public RefreshEndpoint(ContextRefresher contextRefresher) {
        this.contextRefresher = contextRefresher;
    }
     
     //Writes the latest information to the current endpoint
    @WriteOperation
    public Collection<String> refresh() {
       //Refresh the environment and get the latest information
        Set<String> keys = this.contextRefresher.refresh();
        return keys;
    }
}

The specific refresh logic is in contextrefresh.

Configure the ContextRefresher refresh class:

public class ContextRefresher {
	//......
	
	private ConfigurableApplicationContext context;
	private RefreshScope scope;

	public synchronized Set<String> refresh() {
		//Extract previous attribute configuration
		Map<String, Object> before = extract(
				this.context.getEnvironment().getPropertySources());
		//Get the latest property configuration
		addConfigFilesToEnvironment();
		//Get changed properties
		Set<String> keys = changes(before,
				extract(this.context.getEnvironment().getPropertySources())).keySet();
		//Publishing EnvironmentChangeEvent events
		this.context.publishEvent(new EnvironmentChangeEvent(keys));
		//Refresh RefreshScope Bean
		this.scope.refreshAll();
		return keys;
	}
	//......
}

addConfigFilesToEnvironment(); The above code obtains the configuration again through this method:

//Get the latest property configuration
private void addConfigFilesToEnvironment() {
		ConfigurableApplicationContext capture = null;
		try {
			StandardEnvironment environment = copyEnvironment(
					this.context.getEnvironment());
			//Here, recreate the springboot startup class. When restarting, the configuration will be obtained again through the configuration center
			capture = new SpringApplicationBuilder(Empty.class).bannerMode(Mode.OFF)
					.web(false).environment(environment).run();
			if (environment.getPropertySources().contains(REFRESH_ARGS_PROPERTY_SOURCE)) {
				environment.getPropertySources().remove(REFRESH_ARGS_PROPERTY_SOURCE);
			}
			MutablePropertySources target = this.context.getEnvironment()
					.getPropertySources();
			String targetName = null;
		}
	}

Re create the startup class through SpringApplicationBuilder. When starting, the latest configuration will be pulled again, and then the EnvironmentChangeEvent event will be published. The configuration class with @ ConfigurationProperties and the bean with the scope of @ RefreshScope will be reloaded through the corresponding listener.

@RefreshScope

This annotation is an extension type of bean scope made by Spring Cloud. The life cycle of this type of bean is different from that of a single instance. Every time the / refresh method is called, all beans of this type will be cleared. The next time it is used, it will be recreated and the latest attribute variables will be injected.

Let's look at the code.

public class RefreshScope extends GenericScope
		implements ApplicationContextAware, BeanDefinitionRegistryPostProcessor, Ordered {

	//.....
	@ManagedOperation(description = "Dispose of the current instance of all beans in this scope and force a refresh on next method execution.")
	public void refreshAll() {
		//Call the clear cache method
		super.destroy();
		this.context.publishEvent(new RefreshScopeRefreshedEvent());
	}
	//......
}



public class GenericScope implements Scope, BeanFactoryPostProcessor, DisposableBean {

	@Override
	public void destroy() {
		List<Throwable> errors = new ArrayList<Throwable>();
		//Clear cache
		Collection<BeanLifecycleWrapper> wrappers = this.cache.clear();
		for (BeanLifecycleWrapper wrapper : wrappers) {
			try {
			//Destroy all bean s of type @ RefreshScope
				wrapper.destroy();
			}
			catch (RuntimeException e) {
				errors.add(e);
			}
		}
		if (!errors.isEmpty()) {
			throw wrapIfNecessary(errors.get(0));
		}
		this.errors.clear();
	}
}

The destroyed bean will be recreated the next time it is used, which meets the requirements of configuring dynamic refresh. But sometimes, after clearing these beans, you want to execute some custom listening logic. What should you do?

Spring Cloud also provides corresponding events: RefreshScopeRefreshedEvent. After the refreshAll method clears the cache, this event will be published:

this.context.publishEvent(new RefreshScopeRefreshedEvent());

Here are the extensions left. You can make some extensions if necessary. At present, Zuul and Nacos have monitored the event in the source code. Those interested in specific details can study it.

Principle analysis of integrated bus

Old rule: two options, we choose the second

Scheme I:

Scheme II:

The Bus message Bus will provide a / bus/refresh service to realize the hot refresh of applications. The actor is no longer used to provide human refresh logic/ The bus/refresh service requires that the request must be a post request, which can be implemented by any technology.

All ConfigClient instances listen to the same topic in RabbitMQ [SpringCloudBus by default]. When a service refreshes the data, it will put the message into the topic, so that other services listening to the same topic can be notified, and then update their own configuration.

Hot Topics

©2025 Programmer Think