Talk about how to synchronize data to the apollo configuration center

Posted by Or1g1naL on Tue, 22 Feb 2022 02:41:07 +0100

preface

Friends who have landed micro service projects should be familiar with the configuration center. The configuration center can be used for centralized management of configuration and hot update of configuration. At present, the common configuration centers in the market include QConf, spring cloud config, diamond, disconf, apollo, nacos, etc. The most commonly used microservice projects should be spring cloud config, apollo and nacos.

We may have an application scenario where some configuration data is first dropped into the database and then persisted to the configuration center. This can be divided into two steps. The first step is to drop the data into the database, and the second step is to manually write the data to the configuration center through the panel provided by the configuration center. However, we may prefer to synchronize the data directly to the configuration center after the data is stored in the database. Today, take apollo as an example to talk about how to synchronize data to the apollo configuration center

Realization idea

Use the open API provided by apollo to operate

Implementation steps

1. Connect our application to Apollo open platform

Apollo administrator at http: / / {portal_address} / open / manage HTML to create a third-party application, you'd better query whether this AppId has been created before creating it. After successful creation, a token will be generated, as shown in the following figure:

2. Authorize registered apps

Apollo administrator at http: / / {portal_address} / open / manage The HTML page empowers the token. After empowerment, the application can manage the configuration of authorized namespaces through the Http REST interface provided by Apollo

3. Application calls Apollo Open API

Example demonstration

Take the synchronization of API gateway routing information to apollo as an example

1. Create third-party applications

Prompt token after creation

2. appId of the authorization operation to the third-party application according to the token

We are authorized to operate all configurations on the API gateway. The authorization type is APP

3. Call the Apollo Open API through the Apollo Open API

Import Apollo open API coordinates from pom in the project

<dependency>
    <groupId>com.ctrip.framework.apollo</groupId>
    <artifactId>apollo-openapi</artifactId>
    <version>1.7.0</version>
</dependency>

After the introduction, we can directly operate the apollo open api

a. Query configuration item

  public long getMaxRouteRuleIndex(){
        OpenNamespaceDTO openNamespaceDTO = apolloOpenApiClient.getNamespace(appInfoProperties.getAppId(),appInfoProperties.getEnv(),appInfoProperties.getClusterName(),appInfoProperties.getNameSpaceName());
        List<OpenItemDTO> items = openNamespaceDTO.getItems();
        if(CollectionUtils.isEmpty(items)){
            return 0;
        }
        return items.stream().filter(item -> item.getKey().matches(ID_PATTERN)).count();
    }

Run unit test

  @Test
    public void testGetMaxRouteRuleIndex(){
        long index = routeService.getMaxRouteRuleIndex();
        Assert.assertTrue(index >= 0);
    }

The apollo panel on the gateway appears

b. Create and publish configuration items

Note: the creation and publishing of apollo are two different API s

  public boolean createRouteRule(RouteRule routeRule){
        try {
            long curRouteRuleIndex = getMaxRouteRuleIndex();
            buildOpenItemDTO(ROUTE_ID_KEY,curRouteRuleIndex,routeRule.getRouteId(),true);
            buildOpenItemDTO(ROUTE_URI_KEY,curRouteRuleIndex,routeRule.getUri(),true);
            buildOpenItemDTO(ROUTE_PREDICATES_KEY,curRouteRuleIndex,routeRule.getPredicate(),true);
            buildOpenItemDTO(ROUTE_FILTERS_KEY,curRouteRuleIndex,routeRule.getFilter(),true);
            return publish("Add gateway route","Add gateway route");
        } catch (Exception e) {
           log.error("{}",e.getMessage());
        }
        return false;
    }

Run unit test

  @Test
    public void testCreateRouteRule(){
        RouteRule routeRule = RouteRule.builder().routeId(appName)
                .uri("http://localhost:8082")
                .predicate("Path=/dashboard/**")
                .filter("StripPrefix=1").build();

        boolean isSuccess = routeService.createRouteRule(routeRule);
        Assert.assertTrue(isSuccess);
    }

View the panel of api gateway on apollo portal

A route configuration was found. Because the api gateway makes dynamic routing, you can find the following output from the api gateway console

Visit the browser

Dynamic routing takes effect

b. Update and publish configuration items

 public boolean updateRouteRule(RouteRule routeRule){
        long ruleIndex = getRouteRuleIndex(routeRule.getRouteId());
        if(ruleIndex != -1){
            try {
                buildOpenItemDTO(ROUTE_URI_KEY,ruleIndex,routeRule.getUri(),false);
                buildOpenItemDTO(ROUTE_PREDICATES_KEY,ruleIndex,routeRule.getPredicate(),false);
                buildOpenItemDTO(ROUTE_FILTERS_KEY,ruleIndex,routeRule.getFilter(),false);
                return publish("Update gateway route","Update gateway route");
            } catch (Exception e) {
                log.error("{}",e.getMessage());
            }
        }

        return false;
    }

Run unit test

    @Test
    public void testUpdateRouteRule(){
        RouteRule routeRule = RouteRule.builder().routeId(appName)
                .uri("http://localhost:8082")
                .predicate("Path=/xxx/**")
                .filter("StripPrefix=1").build();
        boolean isSuccess = routeService.updateRouteRule(routeRule);
        Assert.assertTrue(isSuccess);
    }

View the panel of api gateway on apollo portal

You can find that the Path of predict has been changed to xxx

View API gateway console

Visit the browser http://localhost:8000/dashboard/ops/index Will appear

Change access http://localhost:8000/xxx/ops/index

This indicates that the route has been changed successfully

b. Delete and publish configuration items

 public boolean deleteRouteRule(String routeId){
        long ruleIndex = getRouteRuleIndex(routeId);

        if(ruleIndex != -1){
            try {
//                removeRouteItem(ROUTE_URI_KEY,ruleIndex);
//                removeRouteItem(ROUTE_PREDICATES_KEY,ruleIndex);
//                removeRouteItem(ROUTE_FILTERS_KEY,ruleIndex);
                buildOpenItemDTO(ROUTE_URI_KEY,ruleIndex,"http://null",false);
                buildOpenItemDTO(ROUTE_PREDICATES_KEY,ruleIndex,"Path=/-9999",false);
                return publish("Delete gateway route","Delete gateway route");
            } catch (Exception e) {
                log.error("{}",e.getMessage());
            }
        }

        return false;
    }
private void removeRouteItem(String key,long index){
        if(key.equalsIgnoreCase(ROUTE_PREDICATES_KEY) || key.equalsIgnoreCase(ROUTE_FILTERS_KEY)){
            key = String.format(key,index,0);
        }else{
            key = String.format(key,index);
        }
        apolloOpenApiClient.removeItem(appInfoProperties.getAppId(),appInfoProperties.getEnv(),appInfoProperties.getClusterName(),appInfoProperties.getNameSpaceName(),key,appInfoProperties.getAuthUser());
    }

Note: because the gateway deletes relatively complex points, which involves recalculation of the route set, it is convenient to update it to an inaccessible route. If it is a physical deletion, just call the removeItem of apollo

summary

The api provided by apollo open platform is actually http restful operation, which provides a series of operations of addition, deletion, modification and query. A small detail here is that the addition, deletion, modification and release of apollo are operated separately. If you only call add, delete and modify, you need to click publish on the portal or use the publish interface to operate. For more details, see the open platform link of apollo

https://www.apolloconfig.com/#/zh/usage/apollo-open-api-platform?id=_3210-%e6%96%b0%e5%a2%9e%e9%85%8d%e7%bd%ae%e6%8e%a5%e5%8f%a3

The examples provided in this article are for reference only and cannot be directly used in production environments. If a friend's configuration center uses nacos, similar operations can be realized. Because nacos also provides an open api interface, interested friends can check the following links

https://nacos.io/zh-cn/docs/open-api.html

demo link

https://github.com/lyb-geek/springboot-learning/tree/master/springboot-sync-apollo

Topics: Java