Sentinel integrates Nacos to realize dynamic rule configuration persistence and two-way synchronization

Posted by Afser on Sun, 06 Mar 2022 02:46:54 +0100

1. Why integrate Nacos

By default, Sentinel configuration rules are stored in memory. After the Sentinel service is re configured, the configuration will show that we realize the persistence of the configuration by integrating third-party middleware, such as using Nacos;

To realize the bidirectional synchronous persistence of Sentinel and Nacos, we need to modify the source package of Sentinel dashboard.

2. Effect demonstration

We take flow control rules as an example to demonstrate the operation of data synchronization and persistence;

1. Syncing nacos to sentinel:

In nacos, add a new configuration file. The DataId of the file is sentinel and the content is:

[
    {
    	"app":"user-service",// Service name
        "resource": "/list", //Resource name
        "count": 1, //threshold 
        "grade": 1, //Threshold type: 0 indicates the number of threads and 1 indicates QPS;
        "limitApp": "default", //Source application	
        "strategy": 0,// Flow control mode, 0 indicates direct, 1 indicates association, and 2 indicates link;
        "controlBehavior": 0 //Flow control effect: 0 indicates fast failure, 1 indicates warm up, and 2 indicates queuing
    }
]


Check Sentinel console: the data has been synchronized

2. sentinel syncs to nacos:

We set up arbitrary flow control rules on sentinel console, as follows:

Check the Nacos console: the configuration data has been synchronized

3. Source pull

1. Download source zip
stay Sentinel-github Download the required version of the compressed package, such as sentinel-1.8.1 zip

2. Load source code
Will download sentinel-1.8.1 Zip, use the IDE tool to open the sentinel dashboard project

3. Modify pom

Comment out the scope tag of sentinel datasource Nacos

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-datasource-nacos</artifactId>
    <!--<scope>test</scope>-->
</dependency>

3. Create public configuration

Before modifying the rule code, you need to create a Nacos configuration file on COM alibaba. csp. sentinel. dashboard. Create a Nacos package under the rule package, and create four classes under the package: RuleNacosConfig, RuleNacosProvider, RuleNacosPublisher and RuleNacosConstants

RuleNacosConfig:

@Configuration
public class RuleNacosConfig{
    @Bean
    public ConfigService nacosConfigService() throws Exception {
        Properties properties = new Properties();
        properties.put(PropertyKeyConst.SERVER_ADDR, "112.15.11.18:8848");
        // properties.put(PropertyKeyConst.NAMESPACE, "xxx");  Namespace
        // properties.put(PropertyKeyConst.USERNAME, "xxx");  user name
        // properties.put(PropertyKeyConst.PASSWORD, "xxx");  password
        return ConfigFactory.createConfigService(properties);
    }
}

RuleNacosProvider:

@Component
public class RuleNacosProvider {

    @Autowired
    private ConfigService configService;

    public String getRules(String dataId, String app) throws Exception {
        // Set the service name to GroupId
        return configService.getConfig(dataId, app, 3000);
    }
}

RuleNacosPublisher:

@Component
public class RuleNacosPublisher {

    @Autowired
    private ConfigService configService;

    public void publish(String dataId, String app, String rules) throws Exception {
        AssertUtil.notEmpty(app, "app name cannot be empty");
        if (rules == null) {
            return;
        }
        // Set the service name to GroupId
        configService.publishConfig(dataId, app, rules);
    }
}

RuleNacosConstants :

public class RuleNacosConstants {
    public static final String FLOW_DATA_ID = "sentinel.rule.flow";
    public static final String DEGRADE_DATA_ID = "sentinel.rule.degrade";
    public static final String SYSTEM_DATA_ID = "sentinel.rule.system";
    public static final String PARAM_DATA_ID = "sentinel.rule.param";
    public static final String AUTHORITY_DATA_ID = "sentinel.rule.authority";
    public static final String GATEWAY_API_DATA_ID = "sentinel.rule.gateway.api";
    public static final String GATEWAY_FLOW_DATA_ID = "sentinel.rule.gateway.flow";
}

4. Console rule configuration

By modifying the source code, the persistence operation of flow control rules, degradation rules, hot spot rules, system rules and authorization rules is realized;

4.1 process rules

1. Modify sidebar html:

<!--take dashboard.flowV1 Change to dashboard.flow -->
<li ui-sref-active="active">
    <a ui-sref="dashboard.flowV1({app: entry.app})">
        <i class="glyphicon glyphicon-filter"></i>&nbsp;&nbsp;Flow control rules
    </a>
</li>

<!--Modified code-->
<li ui-sref-active="active">
    <a ui-sref="dashboard.flow({app: entry.app})">
        <i class="glyphicon glyphicon-filter"></i>&nbsp;&nbsp;Flow control rules
    </a>
</li>

2. Modify FlowControllerV2:
Inject RuleNacosProvider and RuleNacosPublisher into FlowControllerV2

// The modified position is as follows:
@Autowired
@Qualifier("flowRuleDefaultProvider")
private DynamicRuleProvider<List<FlowRuleEntity>> ruleProvider;
@Autowired
@Qualifier("flowRuleDefaultPublisher")
private DynamicRulePublisher<List<FlowRuleEntity>> rulePublisher;

// Change the above code to the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<FlowRuleEntity> rules = ruleProvider.getRules(app);
if (rules != null && !rules.isEmpty()) {
    for (FlowRuleEntity entity : rules) {
         entity.setApp(app);
         if (entity.getClusterConfig() != null && entity.getClusterConfig().getFlowId() != null) {
             entity.setId(entity.getClusterConfig().getFlowId());
          }
     }
}

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.FLOW_DATA_ID, app);
List<FlowRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
    rules = JSON.parseArray(ruleStr, FlowRuleEntity.class);
    if (rules != null && !rules.isEmpty()) {
        for (FlowRuleEntity entity : rules) {
            entity.setApp(app);
        }
    }
 }

Modify push logic:

// The modified position is as follows:
private void publishRules(/*@NonNull*/ String app) throws Exception {
    List<FlowRuleEntity> rules = repository.findAllByApp(app);
    rulePublisher.publish(app, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
  try {
       List<FlowRuleEntity> rules = repository.findAllByApp(app);
       String ruleStr = JSON.toJSONString(rules);
       rulePublisher.publish(RuleNacosConstants.FLOW_DATA_ID, app, ruleStr);
  } catch (Exception e) {
      e.printStackTrace();
  }
}

4.2. Degradation rules

Modify DegradeController:
Inject RuleNacosProvider and RuleNacosPublisher into DegradeController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<DegradeRuleEntity> rules = sentinelApiClient.fetchDegradeRuleOfMachine(app, ip, port);

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.DEGRADE_DATA_ID, app);
List<DegradeRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
    rules = JSON.parseArray(ruleStr, DegradeRuleEntity.class);
    if (rules != null && !rules.isEmpty()) {
       for (DegradeRuleEntity entity : rules) {
           entity.setApp(app);
       }
   }
}

Modify push logic:

// 1. The modified position is as follows:
private boolean publishRules(String app, String ip, Integer port) {
   List<DegradeRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
   return sentinelApiClient.setDegradeRuleOfMachine(app, ip, port, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
  try {
      List<DegradeRuleEntity> rules = repository.findAllByApp(app);
      String ruleStr = JSON.toJSONString(rules);
      rulePublisher.publish(RuleNacosConstants.DEGRADE_DATA_ID, app, ruleStr);
  } catch (Exception e) {
     e.printStackTrace();
  }
}
=======================================================================================

// 2. The modified position is as follows: there are two places
if (!publishRules(entity.getApp(), entity.getIp(), entity.getPort())) {
     logger.warn("Publish degrade rules failed, app={}", entity.getApp());
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 3. The modified position is as follows:
if (!publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
   logger.warn("Publish degrade rules failed, app={}", oldEntity.getApp());
}

// Change the above code to the following code:
publishRules(oldEntity.getApp());

4.3 hot spot rules

Modify ParamRuleController:
Inject RuleNacosProvider and RuleNacosPublisher into ParamFlowRuleController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
return sentinelApiClient.fetchParamFlowRulesOfMachine(app, ip, port)
                .thenApply(repository::saveAll)
                .thenApply(Result::ofSuccess)
                .get();

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.PARAM_DATA_ID, app);
List<ParamFlowRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
    rules = JSON.parseArray(ruleStr, ParamFlowRuleEntity.class);
    if (rules != null && !rules.isEmpty()) {
        for (ParamFlowRuleEntity entity : rules) {
           entity.setApp(app);
   		}
    }
}
rules = repository.saveAll(rules);
return Result.ofSuccess(rules);

Modify push logic:

// 1. The modified position is as follows:
private CompletableFuture<Void> publishRules(String app, String ip, Integer port) {
    List<ParamFlowRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
    return sentinelApiClient.setParamFlowRuleOfMachine(app, ip, port, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
   try {
       List<ParamFlowRuleEntity> rules = repository.findAllByApp(app);
       String ruleStr = JSON.toJSONString(rules);
       rulePublisher.publish(RuleNacosConstants.PARAM_DATA_ID, app, ruleStr);
   } catch (Exception e) {
     e.printStackTrace();
   }
}
=======================================================================================

// 2. The modified position is as follows: there are two places
try {
   entity = repository.save(entity);
   publishRules(entity.getApp(), entity.getIp(), entity.getPort()).get();
   return Result.ofSuccess(entity);
} catch (ExecutionException ex) {
	....
}
// Change the above code to the following code:
try {
   entity = repository.save(entity);
   publishRules(entity.getApp());
   return Result.ofSuccess(entity);
} catch (Exception ex) {
	....
}
=======================================================================================

// 3. The modified position is as follows:
try {
 	repository.delete(id);
    publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort()).get();
    return Result.ofSuccess(id);
} catch (ExecutionException ex) {
	....
}

// Change the above code to the following code:
try {
 	repository.delete(id);
    publishRules(oldEntity.getApp());
    return Result.ofSuccess(id);
} catch (Exception ex) {
	....
}

4.4 system rules

Modify SystemController:
Inject RuleNacosProvider and RuleNacosPublisher into SystemController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<SystemRuleEntity> rules = sentinelApiClient.fetchSystemRuleOfMachine(app, ip, port);

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.SYSTEM_DATA_ID, app);
List<SystemRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
    rules = JSON.parseArray(ruleStr, SystemRuleEntity.class);
    if (rules != null && !rules.isEmpty()) {
        for (SystemRuleEntity entity : rules) {
             entity.setApp(app);
        }
    }
}

Modify push logic:

// 1. The modified position is as follows:
private boolean publishRules(String app, String ip, Integer port) {
   List<SystemRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
   return sentinelApiClient.setSystemRuleOfMachine(app, ip, port, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
  try {
       List<SystemRuleEntity> rules = repository.findAllByApp(app);
       String ruleStr = JSON.toJSONString(rules);
       rulePublisher.publish(RuleNacosConstants.SYSTEM_DATA_ID, app, ruleStr);
   } catch (Exception e) {
       e.printStackTrace();
  }
}
=======================================================================================

// 2. The modified position is as follows
if (!publishRules(app, ip, port)) {
    logger.warn("Publish system rules fail after rule add");
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 3. The modified position is as follows
if (!publishRules(entity.getApp(), entity.getIp(), entity.getPort())) {
    logger.info("publish system rules fail after rule update");
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 4. The modified position is as follows:
if (!publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
    logger.info("publish system rules fail after rule delete");
}

// Change the above code to the following code:
publishRules(oldEntity.getApp());

4.5 authorization rules

To modify the AuthorityRuleController:
Inject RuleNacosPublisher and RuleNacosProvider into AuthorityRuleController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<AuthorityRuleEntity> rules = sentinelApiClient.fetchAuthorityRulesOfMachine(app, ip, port);

// Change the above code to the following code:
 String ruleStr = ruleProvider.getRules(RuleNacosConstants.AUTHORITY_DATA_ID, app);
List<AuthorityRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
     rules = JSON.parseArray(ruleStr, AuthorityRuleEntity.class);
     if (rules != null && !rules.isEmpty()) {
         for (AuthorityRuleEntity entity : rules) {
              entity.setApp(app);
         }
    }
}
  

Push modification logic:

// 1. The modified position is as follows:
private boolean publishRules(String app, String ip, Integer port) {
    List<AuthorityRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
    return sentinelApiClient.setAuthorityRuleOfMachine(app, ip, port, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
   try {
       List<AuthorityRuleEntity> rules = repository.findAllByApp(app);
       String ruleStr = JSON.toJSONString(rules);
       rulePublisher.publish(RuleNacosConstants.AUTHORITY_DATA_ID, app, ruleStr);
   } catch (Exception e) {
       e.printStackTrace();
   }
}
=======================================================================================

// 2. The modified position is as follows: there are two places
if (!publishRules(entity.getApp(), entity.getIp(), entity.getPort())) {
    logger.info("Publish authority rules failed after rule update");
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 3. The modified position is as follows:
if (!publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
   logger.error("Publish authority rules failed after rule delete");
}

// Modify the above code as:
publishRules(oldEntity.getApp());

5. Gateway console rule configuration

To configure the gateway console rule, you need to add a parameter when starting the gateway: - DCSP sentinel. app. type=1

5.1 API management

To modify the GatewayApiController:
Inject RuleNacosPublisher and RuleNacosProvider into GatewayApiController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<ApiDefinitionEntity> apis = sentinelApiClient.fetchApis(app, ip, port).get();

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.GATEWAY_API_DATA_ID, app);
List<ApiDefinitionEntity> apis = new ArrayList<>();
if (ruleStr != null) {
    apis = JSON.parseArray(ruleStr, ApiDefinitionEntity.class);
    if (apis != null && !apis.isEmpty()) {
        for (ApiDefinitionEntity entity : apis) {
             entity.setApp(app);
        }
    }
}
  

Modify push logic:

// 1. The modified position is as follows:
private boolean publishApis(String app, String ip, Integer port) {
   List<ApiDefinitionEntity> apis = repository.findAllByMachine(MachineInfo.of(app, ip, port));
   return sentinelApiClient.modifyApis(app, ip, port, apis);
}

// Change the above code to the following code:
private void publishApi(String app) {
  try {
    	 List<ApiDefinitionEntity> apis= repository.findAllByApp(app);
    	 String ruleStr = JSON.toJSONString(apis);
    	 rulePublisher.publish(RuleNacosConstants.GATEWAY_API_DATA_ID, app, ruleStr);
     } catch (Exception e) {
         e.printStackTrace();
   }
}
=======================================================================================

// 2. The modified position is as follows
if (!publishApis(app, ip, port)) {
     logger.warn("publish gateway apis fail after add");
}

// Change the above code to the following code:
publishApi(entity.getApp());
=======================================================================================

// 3. The modified position is as follows
if (!publishApis(app, entity.getIp(), entity.getPort())) {
    logger.warn("publish gateway apis fail after update");
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 4. The modified position is as follows:
if (!publishApis(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
    logger.warn("publish gateway apis fail after delete");
}

// Change the above code to the following code:
publishRules(oldEntity.getApp());

5.2 process rules

To modify the GatewayFlowRuleController:
Inject RuleNacosPublisher and RuleNacosProvider into GatewayFlowRuleController

// Add the following code:
@Autowired
private RuleNacosProvider ruleProvider;
@Autowired
private RuleNacosPublisher rulePublisher;

Modify read logic:

// The modified position is as follows:
List<GatewayFlowRuleEntity> rules = sentinelApiClient.fetchGatewayFlowRules(app, ip, port).get();

// Change the above code to the following code:
String ruleStr = ruleProvider.getRules(RuleNacosConstants.GATEWAY_FLOW_DATA_ID, app);
List<GatewayFlowRuleEntity> rules = new ArrayList<>();
if (ruleStr != null) {
    rules = JSON.parseArray(ruleStr, GatewayFlowRuleEntity.class);
    if (rules != null && !rules.isEmpty()) {
       for (GatewayFlowRuleEntity entity : rules) {
           entity.setApp(app);
        }
    }
}

Modify push logic:

// 1. The modified position is as follows:
private boolean publishRules(String app, String ip, Integer port) {
   List<GatewayFlowRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
   return sentinelApiClient.modifyGatewayFlowRules(app, ip, port, rules);
}

// Change the above code to the following code:
private void publishRules(String app) {
  try {
      List<GatewayFlowRuleEntity> rules = repository.findAllByApp(app);
      String ruleStr = JSON.toJSONString(rules);
      rulePublisher.publish(RuleNacosConstants.GATEWAY_FLOW_DATA_ID, app, ruleStr);
   } catch (Exception e) {
      e.printStackTrace();
   }
}
=======================================================================================

// 2. The modified position is as follows
if (!publishRules(app, ip, port)) {
    logger.warn("publish gateway flow rules fail after add");
}

// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================

// 3. The modified position is as follows
if (!publishRules(app, entity.getIp(), entity.getPort())) {
    logger.warn("publish gateway flow rules fail after update");
}


// Change the above code to the following code:
publishRules(entity.getApp());
=======================================================================================


// 4. The modified position is as follows:
if (!publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
    logger.warn("publish gateway flow rules fail after delete");
}

// Change the above code to the following code:
publishRules(oldEntity.getApp());

5.3. Degradation rules

The degradation rule interface in console rule configuration is used, and no operation is required. Refer to 4.2 degradation rules.

5.4 system rules

The degradation rule interface in console rule configuration is used, and no operation is required. Refer to 4.4 system rules.

5. Packaged deployment

Enter the destination of sentinel dashboard and package through mvn clean install package -DskipTests=true.

Deployment jar reference:
Building Sentinel console environment with Linux
Docker building Sentinel console environment

6. Source package download

For the above modified code, the source code download address is: https://download.csdn.net/download/zhuocailing3390/83337923 , copy the downloaded Controller and Nacos configuration code directly to the source code.

Topics: Spring Boot Distribution security Nacos sentinel