Nacos source code analysis - configuration center

Posted by NargsBrood on Tue, 04 Jan 2022 19:26:02 +0100

catalogue

Initialize the load configuration process

Configure dynamic refresh process

Client sends long polling request

The server handles long polling requests

summary

Initialize the load configuration process

First, analyze the entry from the startup method of SpringBoot

org.springframework.boot.SpringApplication#run(java.lang.Class<?>[], java.lang.String[])

	public static ConfigurableApplicationContext run(Class<?>[] primarySources, String[] args) {
		return new SpringApplication(primarySources).run(args);
	}

org.springframework.boot.SpringApplication#SpringApplication(org.springframework.core.io.ResourceLoader, java.lang.Class<?>...)

	public SpringApplication(ResourceLoader resourceLoader, Class<?>... primarySources) {
		this.resourceLoader = resourceLoader;
		Assert.notNull(primarySources, "PrimarySources must not be null");
		this.primarySources = new LinkedHashSet<>(Arrays.asList(primarySources));
		this.webApplicationType = WebApplicationType.deduceFromClasspath();
        // Load all classes that implement the ApplicationContextInitializer interface
		setInitializers((Collection) getSpringFactoriesInstances(ApplicationContextInitializer.class));
        // Load all classes that implement the ApplicationListener interface
		setListeners((Collection) getSpringFactoriesInstances(ApplicationListener.class));
		this.mainApplicationClass = deduceMainApplicationClass();
	}

 org.springframework.boot.SpringApplication#run(java.lang.String...)

	public ConfigurableApplicationContext run(String... args) {
		StopWatch stopWatch = new StopWatch();
		stopWatch.start();
		ConfigurableApplicationContext context = null;
		Collection<SpringBootExceptionReporter> exceptionReporters = new ArrayList<>();
		configureHeadlessProperty();
		SpringApplicationRunListeners listeners = getRunListeners(args);
		listeners.starting();
		try {
			ApplicationArguments applicationArguments = new DefaultApplicationArguments(args);
			ConfigurableEnvironment environment = prepareEnvironment(listeners, applicationArguments);
			configureIgnoreBeanInfo(environment);
			Banner printedBanner = printBanner(environment);
			context = createApplicationContext();
			exceptionReporters = getSpringFactoriesInstances(SpringBootExceptionReporter.class,
					new Class[] { ConfigurableApplicationContext.class }, context);
			prepareContext(context, environment, listeners, applicationArguments, printedBanner);
			refreshContext(context);
			afterRefresh(context, applicationArguments);
			stopWatch.stop();
			if (this.logStartupInfo) {
				new StartupInfoLogger(this.mainApplicationClass).logStarted(getApplicationLog(), stopWatch);
			}
			listeners.started(context);
			callRunners(context, applicationArguments);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, listeners);
			throw new IllegalStateException(ex);
		}

		try {
			listeners.running(context);
		}
		catch (Throwable ex) {
			handleRunFailure(context, ex, exceptionReporters, null);
			throw new IllegalStateException(ex);
		}
		return context;
	}

In the run method, we focus on the prepareContext method. There is an applyInitializers method in this method

org.springframework.boot.SpringApplication#applyInitializers

	protected void applyInitializers(ConfigurableApplicationContext context) {
        // During initialization, load all classes that implement the ApplicationContextInitializer interface and execute the initialize method
		for (ApplicationContextInitializer initializer : getInitializers()) {
			Class<?> requiredType = GenericTypeResolver.resolveTypeArgument(initializer.getClass(),
					ApplicationContextInitializer.class);
			Assert.isInstanceOf(requiredType, context, "Unable to call initializer.");
			initializer.initialize(context);
		}
	}

In the process of starting the service, you can see from the console that the propertysourcebootstrap configuration class reads the corresponding configuration of Nacos, and this class also implements the ApplicationContextInitializer interface. Let's take a look at its initialize method.

org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration#initialize

	public void initialize(ConfigurableApplicationContext applicationContext) {
		List<PropertySource<?>> composite = new ArrayList<>();
		AnnotationAwareOrderComparator.sort(this.propertySourceLocators);
		boolean empty = true;
		ConfigurableEnvironment environment = applicationContext.getEnvironment();
        // Traverse all implementation classes that implement the PropertySourceLocator interface (under the dependency of using Nacos as the configuration center, its implementation class is only NacosPropertySourceLocator)
		for (PropertySourceLocator locator : this.propertySourceLocators) {
            // This is to call the locate method of the implementation class,
			Collection<PropertySource<?>> source = locator.locateCollection(environment);
			if (source == null || source.size() == 0) {
				continue;
			}
			List<PropertySource<?>> sourceList = new ArrayList<>();
			for (PropertySource<?> p : source) {
				sourceList.add(new BootstrapPropertySource<>(p));
			}
			logger.info("Located property source: " + sourceList);
			composite.addAll(sourceList);
			empty = false;
		}
		if (!empty) {
			MutablePropertySources propertySources = environment.getPropertySources();
			String logConfig = environment.resolvePlaceholders("${logging.config:}");
			LogFile logFile = LogFile.get(environment);
			for (PropertySource<?> p : environment.getPropertySources()) {
				if (p.getName().startsWith(BOOTSTRAP_PROPERTY_SOURCE_NAME)) {
					propertySources.remove(p.getName());
				}
			}
			insertPropertySources(propertySources, composite);
			reinitializeLoggingSystem(environment, logConfig, logFile);
			setLogLevels(applicationContext, environment);
			handleIncludedProfiles(environment);
		}
	}

com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#locate

	public PropertySource<?> locate(Environment env) {
		nacosConfigProperties.setEnvironment(env);
        // Create NacosConfigService through reflection
		ConfigService configService = nacosConfigManager.getConfigService();

		if (null == configService) {
			log.warn("no instance of config service found, can't load config from nacos");
			return null;
		}
		long timeout = nacosConfigProperties.getTimeout();
		nacosPropertySourceBuilder = new NacosPropertySourceBuilder(configService,
				timeout);
        // Get the value of name
		String name = nacosConfigProperties.getName();
        // Get the value of prefix
		String dataIdPrefix = nacosConfigProperties.getPrefix();
		if (StringUtils.isEmpty(dataIdPrefix)) {
            // If prefix is empty, dataIdPrefix is the value of name
			dataIdPrefix = name;
		}

		if (StringUtils.isEmpty(dataIdPrefix)) {
            // If it is judged to be empty again, spring. Com is directly obtained application. Value of name
			dataIdPrefix = env.getProperty("spring.application.name");
		}

		CompositePropertySource composite = new CompositePropertySource(
				NACOS_PROPERTY_SOURCE_NAME);
        // Load shared configuration
		loadSharedConfiguration(composite);
        // Load extended configuration
		loadExtConfiguration(composite);
        // Load current application configuration
		loadApplicationConfiguration(composite, dataIdPrefix, nacosConfigProperties, env);

		return composite;
	}

 com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadSharedConfiguration

	private void loadSharedConfiguration(
			CompositePropertySource compositePropertySource) {
		List<NacosConfigProperties.Config> sharedConfigs = nacosConfigProperties
				.getSharedConfigs();
        // Load shared configurations (i.e. configure shared configurations)
		if (!CollectionUtils.isEmpty(sharedConfigs)) {
			checkConfiguration(sharedConfigs, "shared-configs");
			loadNacosConfiguration(compositePropertySource, sharedConfigs);
		}
	}

com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadExtConfiguration

	private void loadExtConfiguration(CompositePropertySource compositePropertySource) {
		List<NacosConfigProperties.Config> extConfigs = nacosConfigProperties
				.getExtensionConfigs();
        // Load extension configuration (i.e. configure extension configurations)
		if (!CollectionUtils.isEmpty(extConfigs)) {
			checkConfiguration(extConfigs, "extension-configs");
			loadNacosConfiguration(compositePropertySource, extConfigs);
		}
	}

com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadApplicationConfiguration

	private void loadApplicationConfiguration(
			CompositePropertySource compositePropertySource, String dataIdPrefix,
			NacosConfigProperties properties, Environment environment) {
        // Get the file suffix, i.e. file extension (default properties)
		String fileExtension = properties.getFileExtension();
        // Get the group name, i.e. group (DEFAULT_GROUP)
		String nacosGroup = properties.getGroup();
		// load directly once by default
        // Load dataIdPrefix as the configuration of dataId
		loadNacosDataIfPresent(compositePropertySource, dataIdPrefix, nacosGroup,
				fileExtension, true);
		// load with suffix, which have a higher priority than the default
        // Load dataIdPrefix + fileExtension as the configuration of dataId
		loadNacosDataIfPresent(compositePropertySource,
				dataIdPrefix + DOT + fileExtension, nacosGroup, fileExtension, true);
		// Loaded with profile, which have a higher priority than the suffix
		for (String profile : environment.getActiveProfiles()) {
			String dataId = dataIdPrefix + SEP1 + profile + DOT + fileExtension;
            // Load dataIdPrefix + profile + fileExtension as the configuration of dataId
			loadNacosDataIfPresent(compositePropertySource, dataId, nacosGroup,
					fileExtension, true);
		}

	}

It can be seen that the order of configuration loading is shared configuration - > extension configuration - > application configuration. When loading application configuration, the specific priority order is prefix + environment + suffix > prefix + suffix > prefix (Note: the prefix is dataIdPrefix, the environment is profile, and the suffix is fileExtension).

The three loading methods are loadNacosDataIfPresent

com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosDataIfPresent

	private void loadNacosDataIfPresent(final CompositePropertySource composite,
			final String dataId, final String group, String fileExtension,
			boolean isRefreshable) {
		if (null == dataId || dataId.trim().length() < 1) {
			return;
		}
		if (null == group || group.trim().length() < 1) {
			return;
		}
		NacosPropertySource propertySource = this.loadNacosPropertySource(dataId, group,
				fileExtension, isRefreshable);
		this.addFirstPropertySource(composite, propertySource, false);
	}

 com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosPropertySource

	private NacosPropertySource loadNacosPropertySource(final String dataId,
			final String group, String fileExtension, boolean isRefreshable) {
		if (NacosContextRefresher.getRefreshCount() != 0) {
			if (!isRefreshable) {
                // No dynamic refresh is required, and it is obtained directly from the local cache
				return NacosPropertySourceRepository.getNacosPropertySource(dataId,
						group);
			}
		}
        // Get from remote
		return nacosPropertySourceBuilder.build(dataId, group, fileExtension,
				isRefreshable);
	}

The process of obtaining the local cache is to obtain the dataid and group from the Map as the key. Here we focus on the method of obtaining from the remote.

com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder#build

	NacosPropertySource build(String dataId, String group, String fileExtension,
			boolean isRefreshable) {
        // load configuration
		Map<String, Object> p = loadNacosData(dataId, group, fileExtension);
		NacosPropertySource nacosPropertySource = new NacosPropertySource(group, dataId,
				p, new Date(), isRefreshable);
        // Put into local cache
		NacosPropertySourceRepository.collectNacosPropertySource(nacosPropertySource);
		return nacosPropertySource;
	}

com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder#loadNacosData

	private Map<String, Object> loadNacosData(String dataId, String group,
			String fileExtension) {
		String data = null;
		try {
            // The specific execution method is the getConfigInner method of the NacosConfigService class
			data = configService.getConfig(dataId, group, timeout);
			if (StringUtils.isEmpty(data)) {
				log.warn(
						"Ignore the empty nacos configuration and get it based on dataId[{}] & group[{}]",
						dataId, group);
				return EMPTY_MAP;
			}
			if (log.isDebugEnabled()) {
				log.debug(String.format(
						"Loading nacos data, dataId: '%s', group: '%s', data: %s", dataId,
						group, data));
			}
			Map<String, Object> dataMap = NacosDataParserHandler.getInstance()
					.parseNacosData(data, fileExtension);
			return dataMap == null ? EMPTY_MAP : dataMap;
		}
		catch (NacosException e) {
			log.error("get data from Nacos error,dataId:{}, ", dataId, e);
		}
		catch (Exception e) {
			log.error("parse data from Nacos error,dataId:{},data:{},", dataId, data, e);
		}
		return EMPTY_MAP;
	}

com.alibaba.nacos.client.config.NacosConfigService#getConfigInner

    private String getConfigInner(String tenant, String dataId, String group, long timeoutMs) throws NacosException {
        group = null2defaultGroup(group);
        ParamUtils.checkKeyParam(dataId, group);
        ConfigResponse cr = new ConfigResponse();

        cr.setDataId(dataId);
        cr.setTenant(tenant);
        cr.setGroup(group);

        // Give priority to local configuration and try to get (/ data / config data) from the specified directory. By default, the path is ${user.home}/nacos/config/
        String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
        if (content != null) {
            LOGGER.warn("[{}] [get-config] get failover ok, dataId={}, group={}, tenant={}, config={}", agent.getName(),
                dataId, group, tenant, ContentUtils.truncateContent(content));
            cr.setContent(content);
            configFilterChainManager.doFilter(null, cr);
            content = cr.getContent();
            return content;
        }

        try {
            // If the local configuration is not obtained, call the server to obtain it (/ v1/cs/configs)
            String[] ct = worker.getServerConfig(dataId, group, tenant, timeoutMs);
            cr.setContent(ct[0]);

            configFilterChainManager.doFilter(null, cr);
            content = cr.getContent();

            return content;
        } catch (NacosException ioe) {
            if (NacosException.NO_RIGHT == ioe.getErrCode()) {
                throw ioe;
            }
            LOGGER.warn("[{}] [get-config] get from server error, dataId={}, group={}, tenant={}, msg={}",
                agent.getName(), dataId, group, tenant, ioe.toString());
        }

        LOGGER.warn("[{}] [get-config] get snapshot ok, dataId={}, group={}, tenant={}, config={}", agent.getName(),
            dataId, group, tenant, ContentUtils.truncateContent(content));
        // If it is not obtained from the server, it is directly in the form of local snapshot (/ snapshot)
        content = LocalConfigInfoProcessor.getSnapshot(agent.getName(), dataId, group, tenant);
        cr.setContent(content);
        configFilterChainManager.doFilter(null, cr);
        content = cr.getContent();
        return content;
    }

The above is a process of loading configuration when the client starts, but how does the client refresh dynamically when the configuration information changes?

Configure dynamic refresh process

Client sends long polling request

When the client starts loading the configuration, you can see it on COM alibaba. cloud. nacos. client. In the method of nacospropertysourcelocator #locate, NacosConfigService was first created by using reflection. Here we take a look at its constructor

com.alibaba.nacos.client.config.NacosConfigService#NacosConfigService

    public NacosConfigService(Properties properties) throws NacosException {
        String encodeTmp = properties.getProperty(PropertyKeyConst.ENCODE);
        if (StringUtils.isBlank(encodeTmp)) {
            encode = Constants.ENCODE;
        } else {
            encode = encodeTmp.trim();
        }
        initNamespace(properties);
        // MetricsHttpAgen is encapsulated by decorator mode. The real processing is ServerHttpAgent, which uses a thread pool with 1 core threads to call the login interface of the server every 5 seconds (/ v1/auth/users/login)
        agent = new MetricsHttpAgent(new ServerHttpAgent(properties));
        agent.start();
        // Create a ClientWorker object
        worker = new ClientWorker(agent, configFilterChainManager, properties);
    }

Here's how to create ClientWorker.

com.alibaba.nacos.client.config.impl.ClientWorker#ClientWorker

    public ClientWorker(final HttpAgent agent, final ConfigFilterChainManager configFilterChainManager, final Properties properties) {
        this.agent = agent;
        this.configFilterChainManager = configFilterChainManager;

        // Initialize the timeout parameter

        init(properties);
        // Initialize a thread pool with 1 core threads
        executor = Executors.newScheduledThreadPool(1, new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                Thread t = new Thread(r);
                t.setName("com.alibaba.nacos.client.Worker." + agent.getName());
                t.setDaemon(true);
                return t;
            }
        });
        // Initialize the thread pool for long polling. The number of core threads is runtime getRuntime(). availableProcessors()
        executorService = Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                Thread t = new Thread(r);
                t.setName("com.alibaba.nacos.client.Worker.longPolling." + agent.getName());
                t.setDaemon(true);
                return t;
            }
        });
        // The first delay is 1s, and the subsequent delay is 10s. Check the configuration information
        executor.scheduleWithFixedDelay(new Runnable() {
            @Override
            public void run() {
                try {
                    checkConfigInfo();
                } catch (Throwable e) {
                    LOGGER.error("[" + agent.getName() + "] [sub-check] rotate check error", e);
                }
            }
        }, 1L, 10L, TimeUnit.MILLISECONDS);
    }

com.alibaba.nacos.client.config.impl.ClientWorker#checkConfigInfo

    public void checkConfigInfo() {
        // Task fragmentation
        int listenerSize = cacheMap.get().size();
        // Round up to the number of batches to ensure that each task can be checked (paramutil. Getpertask configsize() defaults to 3000. Assuming there are 4000 configurations, it will be divided into two pieces)
        int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
        if (longingTaskCount > currentLongingTaskCount) {
            for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) {
                // Execute the task (LongPollingRunnable implements Runnable, and the specific execution method is the run method)
                executorService.execute(new LongPollingRunnable(i));
            }
            currentLongingTaskCount = longingTaskCount;
        }
    }

com.alibaba.nacos.client.config.impl.ClientWorker.LongPollingRunnable#run

        public void run() {

            List<CacheData> cacheDatas = new ArrayList<CacheData>();
            List<String> inInitializingCacheList = new ArrayList<String>();
            try {
                // check failover config
                for (CacheData cacheData : cacheMap.get().values()) {
                    // Only the configuration of the current slice is processed
                    if (cacheData.getTaskId() == taskId) {
                        cacheDatas.add(cacheData);
                        try {
                            // Check local configuration
                            checkLocalConfig(cacheData);
                            // Judge whether to use local configuration. If true, perform md5 comparison
                            if (cacheData.isUseLocalConfigInfo()) {
                                cacheData.checkListenerMd5();
                            }
                        } catch (Exception e) {
                            LOGGER.error("get local config info error", e);
                        }
                    }
                }

                // Check whether the configuration under the current partition has changed on the server, and return the changed key,
                List<String> changedGroupKeys = checkUpdateDataIds(cacheDatas, inInitializingCacheList);
                LOGGER.info("get changedGroupKeys:" + changedGroupKeys);
                // Traverse the changed key, request the server interface to obtain the latest configuration according to the key, and update it to the cache
                for (String groupKey : changedGroupKeys) {
                    String[] key = GroupKey.parseKey(groupKey);
                    String dataId = key[0];
                    String group = key[1];
                    String tenant = null;
                    if (key.length == 3) {
                        tenant = key[2];
                    }
                    try {
                        String[] ct = getServerConfig(dataId, group, tenant, 3000L);
                        CacheData cache = cacheMap.get().get(GroupKey.getKeyTenant(dataId, group, tenant));
                        cache.setContent(ct[0]);
                        if (null != ct[1]) {
                            cache.setType(ct[1]);
                        }
                        LOGGER.info("[{}] [data-received] dataId={}, group={}, tenant={}, md5={}, content={}, type={}",
                            agent.getName(), dataId, group, tenant, cache.getMd5(),
                            ContentUtils.truncateContent(ct[0]), ct[1]);
                    } catch (NacosException ioe) {
                        String message = String.format(
                            "[%s] [get-update] get changed config exception. dataId=%s, group=%s, tenant=%s",
                            agent.getName(), dataId, group, tenant);
                        LOGGER.error(message, ioe);
                    }
                }
                // Traverse the configuration under the partition. If the isInitializing of the configuration is false (that is, it does not appear in the cacheMap for the first time) or the configuration exists in the ininitializing cachelist, compare md5 and set isInitializing to false
                for (CacheData cacheData : cacheDatas) {
                    if (!cacheData.isInitializing() || inInitializingCacheList
                        .contains(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant))) {
                        cacheData.checkListenerMd5();
                        cacheData.setInitializing(false);
                    }
                }
                // Clear first cache configuration list
                inInitializingCacheList.clear();
                // Continue with the task
                executorService.execute(this);

            } catch (Throwable e) {

                // If the rotation training task is abnormal, the next execution time of the task will be punished
                // If an exception occurs, the next round of task execution will be extended
                LOGGER.error("longPolling error : ", e);
                executorService.schedule(this, taskPenaltyTime, TimeUnit.MILLISECONDS);
            }
        }

com.alibaba.nacos.client.config.impl.ClientWorker#checkLocalConfig

    private void checkLocalConfig(CacheData cacheData) {
        final String dataId = cacheData.dataId;
        final String group = cacheData.group;
        final String tenant = cacheData.tenant;
        // Locally configured files
        File path = LocalConfigInfoProcessor.getFailoverFile(agent.getName(), dataId, group, tenant);

        // If the local configuration is not used and the local configuration file exists, the local configuration is used and isUseLocalConfig is set to true (that is, the local configuration is used)
        if (!cacheData.isUseLocalConfigInfo() && path.exists()) {
            String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
            String md5 = MD5.getInstance().getMD5String(content); 
            cacheData.setUseLocalConfigInfo(true);
            cacheData.setLocalConfigInfoVersion(path.lastModified());
            cacheData.setContent(content);

            LOGGER.warn("[{}] [failover-change] failover file created. dataId={}, group={}, tenant={}, md5={}, content={}",
                agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
            return;
        }

        // If the local configuration is used and the local configuration file does not exist, directly set isUseLocalConfig to false (do not use the local configuration)
        if (cacheData.isUseLocalConfigInfo() && !path.exists()) {
            cacheData.setUseLocalConfigInfo(false);
            LOGGER.warn("[{}] [failover-change] failover file deleted. dataId={}, group={}, tenant={}", agent.getName(),
                dataId, group, tenant);
            return;
        }

        // When the local configuration is used and the locally configured file exists and the version in the current memory is inconsistent with that of the local file, use the configuration of the local file and set the version in memory to the version of the local file
        if (cacheData.isUseLocalConfigInfo() && path.exists()
            && cacheData.getLocalConfigInfoVersion() != path.lastModified()) {
            String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
            String md5 = MD5.getInstance().getMD5String(content);
            cacheData.setUseLocalConfigInfo(true);
            cacheData.setLocalConfigInfoVersion(path.lastModified());
            cacheData.setContent(content);
            LOGGER.warn("[{}] [failover-change] failover file changed. dataId={}, group={}, tenant={}, md5={}, content={}",
                agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
        }
    }

com.alibaba.nacos.client.config.impl.ClientWorker#checkUpdateDataIds

    List<String> checkUpdateDataIds(List<CacheData> cacheDatas, List<String> inInitializingCacheList) throws IOException {
        StringBuilder sb = new StringBuilder();
        // Traverse the configuration information in the cache under the partition and splice the data that does not use the local configuration
        for (CacheData cacheData : cacheDatas) {
            if (!cacheData.isUseLocalConfigInfo()) {
                sb.append(cacheData.dataId).append(WORD_SEPARATOR);
                sb.append(cacheData.group).append(WORD_SEPARATOR);
                if (StringUtils.isBlank(cacheData.tenant)) {
                    sb.append(cacheData.getMd5()).append(LINE_SEPARATOR);
                } else {
                    sb.append(cacheData.getMd5()).append(WORD_SEPARATOR);
                    sb.append(cacheData.getTenant()).append(LINE_SEPARATOR);
                }
                if (cacheData.isInitializing()) {
                    // cacheData appears in the cacheMap for the first time and is updated by check for the first time
                    inInitializingCacheList
                        .add(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant));
                }
            }
        }
        boolean isInitializingCacheList = !inInitializingCacheList.isEmpty();
        // Check data that does not use local configuration
        return checkUpdateConfigStr(sb.toString(), isInitializingCacheList);
    }

com.alibaba.nacos.client.config.impl.ClientWorker#checkUpdateConfigStr

    List<String> checkUpdateConfigStr(String probeUpdateString, boolean isInitializingCacheList) throws IOException {


        List<String> params = new ArrayList<String>(2);
        params.add(Constants.PROBE_MODIFY_REQUEST);
        params.add(probeUpdateString);

        List<String> headers = new ArrayList<String>(2);
        headers.add("Long-Pulling-Timeout");
        headers.add("" + timeout);

        // told server do not hang me up if new initializing cacheData added in
        if (isInitializingCacheList) {
            headers.add("Long-Pulling-Timeout-No-Hangup");
            headers.add("true");
        }

        if (StringUtils.isBlank(probeUpdateString)) {
            return Collections.emptyList();
        }

        try {
            // In order to prevent the server from handling the delay of the client's long task,
            // increase the client's read timeout to avoid this problem.
            // Initiate a long polling request. The default timeout is 45S. The request path is: / nacos/v1/ns/configs/listener
            long readTimeoutMs = timeout + (long) Math.round(timeout >> 1);
            HttpResult result = agent.httpPost(Constants.CONFIG_CONTROLLER_PATH + "/listener", headers, params,
                agent.getEncode(), readTimeoutMs);

            if (HttpURLConnection.HTTP_OK == result.code) {
                setHealthServer(true);
                // If successful, the response content will be parsed and the changed groupKey will be returned
                return parseUpdateDataIdResponse(result.content);
            } else {
                setHealthServer(false);
                LOGGER.error("[{}] [check-update] get changed dataId error, code: {}", agent.getName(), result.code);
            }
        } catch (IOException e) {
            setHealthServer(false);
            LOGGER.error("[" + agent.getName() + "] [check-update] get changed dataId exception", e);
            throw e;
        }
        return Collections.emptyList();
    }

The server handles long polling requests

com.alibaba.nacos.config.server.controller.ConfigController#listener

    public void listener(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {
        request.setAttribute("org.apache.catalina.ASYNC_SUPPORTED", true);
        // String spliced without local configuration information
        String probeModify = request.getParameter("Listening-Configs");
        if (StringUtils.isBlank(probeModify)) {
            throw new IllegalArgumentException("invalid probeModify");
        }

        log.info("listen config id:" + probeModify);

        probeModify = URLDecoder.decode(probeModify, Constants.ENCODE);

        Map<String, String> clientMd5Map;
        try {
            // Parse string
            clientMd5Map = MD5Util.getClientMd5Map(probeModify);
        } catch (Throwable e) {
            throw new IllegalArgumentException("invalid probeModify");
        }

        log.info("listen config id 2:" + probeModify);

        // do long-polling
        // Execute long polling
        inner.doPollingConfig(request, response, clientMd5Map, probeModify.length());
    }

com.alibaba.nacos.config.server.controller.ConfigServletInner#doPollingConfig

    public String doPollingConfig(HttpServletRequest request, HttpServletResponse response,
                                  Map<String, String> clientMd5Map, int probeRequestSize)
        throws IOException {

        // Judge whether there is long pulling timeout according to the request header. If so, it is long polling
        if (LongPollingService.isSupportLongPolling(request)) {
            // Long polling, blocking the return of requests
            longPollingService.addLongPollingClient(request, response, clientMd5Map, probeRequestSize);
            return HttpServletResponse.SC_OK + "";
        }

        // Short polling: traverse the configured md5 and return the changed groupKey
        List<String> changedGroups = MD5Util.compareMd5(request, response, clientMd5Map);

        // Compatible short polling result
        String oldResult = MD5Util.compareMd5OldResult(changedGroups);
        String newResult = MD5Util.compareMd5ResultString(changedGroups);

        String version = request.getHeader(Constants.CLIENT_VERSION_HEADER);
        if (version == null) {
            version = "2.0.0";
        }
        int versionNum = Protocol.getVersionNumber(version);

        /**
         * 2.0.4 Before version, the return value is placed in the header
         */
        if (versionNum < START_LONGPOLLING_VERSION_NUM) {
            response.addHeader(Constants.PROBE_MODIFY_RESPONSE, oldResult);
            response.addHeader(Constants.PROBE_MODIFY_RESPONSE_NEW, newResult);
        } else {
            request.setAttribute("content", newResult);
        }

        Loggers.AUTH.info("new content:" + newResult);

        // disable cache 
        response.setHeader("Pragma", "no-cache");
        response.setDateHeader("Expires", 0);
        response.setHeader("Cache-Control", "no-cache,no-store");
        response.setStatus(HttpServletResponse.SC_OK);
        return HttpServletResponse.SC_OK + "";
    }

The method of long polling plus short polling is implemented here. For short polling directly called by ordinary requests, the results are directly returned in the content and directly parsed by the client. See the long polling method below

com.alibaba.nacos.config.server.service.LongPollingService#addLongPollingClient

    public void addLongPollingClient(HttpServletRequest req, HttpServletResponse rsp, Map<String, String> clientMd5Map,
                                     int probeRequestSize) {

        String str = req.getHeader(LongPollingService.LONG_POLLING_HEADER);
        String noHangUpFlag = req.getHeader(LongPollingService.LONG_POLLING_NO_HANG_UP_HEADER);
        String appName = req.getHeader(RequestUtil.CLIENT_APPNAME_HEADER);
        String tag = req.getHeader("Vipserver-Tag");
        int delayTime = SwitchService.getSwitchInteger(SwitchService.FIXED_DELAY_TIME, 500);
        /**
         * Return the response 500ms in advance to avoid client timeout @ Qiaoyi dingqy 2013.10. 22 add delay time for LoadBalance
         */
        long timeout = Math.max(10000, Long.parseLong(str) - delayTime);
        if (isFixedPolling()) {
            timeout = Math.max(10000, getFixedPollingInterval());
            // do nothing but set fix polling timeout
        } else {
            long start = System.currentTimeMillis();
            // By comparing md5, the changed groupKey is returned
            List<String> changedGroups = MD5Util.compareMd5(req, rsp, clientMd5Map);
            // If the key list is greater than 0, the data will be returned directly to end the long polling
            if (changedGroups.size() > 0) {
                generateResponse(req, rsp, changedGroups);
                LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}",
                    System.currentTimeMillis() - start, "instant", RequestUtil.getRemoteIp(req), "polling",
                    clientMd5Map.size(), probeRequestSize, changedGroups.size());
                return;
            } else if (noHangUpFlag != null && noHangUpFlag.equalsIgnoreCase(TRUE_STR)) {
                LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}", System.currentTimeMillis() - start, "nohangup",
                    RequestUtil.getRemoteIp(req), "polling", clientMd5Map.size(), probeRequestSize,
                    changedGroups.size());
                return;
            }
        }
        String ip = RequestUtil.getRemoteIp(req);
        // It must be called by the HTTP thread, or the container will send a response immediately after leaving
        final AsyncContext asyncContext = req.startAsync();
        // AsyncContext. The timeout of settimeout() is not allowed, so you can only control it yourself
        asyncContext.setTimeout(0L);
        // Start a new long polling thread
        scheduler.execute(
            new ClientLongPolling(asyncContext, clientMd5Map, ip, probeRequestSize, timeout, appName, tag));
    }

The main logic here is as follows:

1. Read the corresponding data from the request header and calculate the timeout, which is 30000-500ms by default to ensure that the response can be given to the client before the timeout

2. Judge whether isFixedPolling is true, and if it is true, recalculate the timeout; Go to point 3 for false

3. If isFixedPolling is false, the configuration is compared through md5. If there is a changed configuration, it is directly returned to the client to end the long polling; No change configuration, go to point 4

4. If the configuration is not modified, it is judged that the long pulling timeout no hangup of the request header is not empty and true, and it is returned directly

5. If none of the above logic returns, AsyncContext asynchronous processing is created through the request, and a new long polling task is created through the thread pool

com.alibaba.nacos.config.server.service.LongPollingService.ClientLongPolling#run

        public void run() {
            asyncTimeoutFuture = scheduler.schedule(new Runnable() {
                @Override
                public void run() {
                    try {
                        getRetainIps().put(ClientLongPolling.this.ip, System.currentTimeMillis());
                        // Remove the current long polling instance from the queue
                        allSubs.remove(ClientLongPolling.this);

                        if (isFixedPolling()) {
                            // If it is a fixed long polling, find the updated groupKey through md5 comparison, and return
                            LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}",
                                (System.currentTimeMillis() - createTime),
                                "fix", RequestUtil.getRemoteIp((HttpServletRequest)asyncContext.getRequest()),
                                "polling",
                                clientMd5Map.size(), probeRequestSize);
                            List<String> changedGroups = MD5Util.compareMd5(
                                (HttpServletRequest)asyncContext.getRequest(),
                                (HttpServletResponse)asyncContext.getResponse(), clientMd5Map);
                            if (changedGroups.size() > 0) {
                                sendResponse(changedGroups);
                            } else {
                                sendResponse(null);
                            }
                        } else {
                            LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}",
                                (System.currentTimeMillis() - createTime),
                                "timeout", RequestUtil.getRemoteIp((HttpServletRequest)asyncContext.getRequest()),
                                "polling",
                                clientMd5Map.size(), probeRequestSize);
                            sendResponse(null);
                        }
                    } catch (Throwable t) {
                        LogUtil.defaultLog.error("long polling error:" + t.getMessage(), t.getCause());
                    }

                }

            }, timeoutTime, TimeUnit.MILLISECONDS);
            // Add the long polling instance to the queue
            allSubs.add(this);
        }

It can be seen from this code that this long polling task is a delayed task. If the configuration changes during the delayed period, how can the client dynamically perceive it? My guess is that there will be event monitoring.

// LongPollingService inherits AbstractEventListener and adds listeners by rewriting
public class LongPollingService extends AbstractEventListener {

    public List<Class<? extends Event>> interest() {
        List<Class<? extends Event>> eventTypes = new ArrayList<Class<? extends Event>>();
        // Added LocalDataChangeEvent listener
        eventTypes.add(LocalDataChangeEvent.class);
        return eventTypes;
    }

    
    public void onEvent(Event event) {
        if (isFixedPolling()) {
            // ignore
        } else {
            if (event instanceof LocalDataChangeEvent) {
                LocalDataChangeEvent evt = (LocalDataChangeEvent)event;
                // Execute the DataChangeTask task through the thread pool
                scheduler.execute(new DataChangeTask(evt.groupKey, evt.isBeta, evt.betaIps));
            }
        }
    }
   
}

com.alibaba.nacos.config.server.service.LongPollingService.DataChangeTask#run

        public void run() {
            try {
                ConfigService.getContentBetaMd5(groupKey);
                // Traverse the long polling instance queue
                for (Iterator<ClientLongPolling> iter = allSubs.iterator(); iter.hasNext(); ) {
                    ClientLongPolling clientSub = iter.next();
                    // Long polling instance found matching
                    if (clientSub.clientMd5Map.containsKey(groupKey)) {
                        // If the beta is released and is not in the beta list, skip it directly
                        if (isBeta && !betaIps.contains(clientSub.ip)) {
                            continue;
                        }

                        // If the tag is published and not in the tag list, skip it directly
                        if (StringUtils.isNotBlank(tag) && !tag.equals(clientSub.tag)) {
                            continue;
                        }

                        getRetainIps().put(clientSub.ip, System.currentTimeMillis());
                        // Remove the long polling instance from the allSubs queue
                        iter.remove();
                        LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}",
                            (System.currentTimeMillis() - changeTime),
                            "in-advance",
                            RequestUtil.getRemoteIp((HttpServletRequest)clientSub.asyncContext.getRequest()),
                            "polling",
                            clientSub.clientMd5Map.size(), clientSub.probeRequestSize, groupKey);
                        // Send the return information (judge whether the delayed scheduling task is cancelled, if not, cancel it first, and then return the data)
                        clientSub.sendResponse(Arrays.asList(groupKey));
                    }
                }
            } catch (Throwable t) {
                LogUtil.defaultLog.error("data change error:" + t.getMessage(), t.getCause());
            }
        }

On the server side, you can see a series of isFixedPolling judgments. This means fixed long polling (client pull mode)

summary

By viewing the source code of the entire configuration center, it can be seen that Nacos configuration center realizes the pull of the client and the push of the server. The specific steps are as follows:

1. Client initiated request

2. After receiving the request, the server compares whether it has changed through md5; If there is a change, directly return the data and return to step 1. If there is no change, directly suspend the request

3. If the data has not changed during this period, the request is ended; If there is a change, the server will poll and listen to the configuration item request instance and directly return data

It is worth noting that the server will not return content here, but only groupKey(dataId+groupId)

 

Topics: Java Spring Spring Boot