catalogue
Initialize the load configuration process
Configure dynamic refresh process
Client sends long polling request
The server handles long polling requests
Initialize the load configuration process
First, analyze the entry from the startup method of SpringBoot
org.springframework.boot.SpringApplication#run(java.lang.Class<?>[], java.lang.String[])
public static ConfigurableApplicationContext run(Class<?>[] primarySources, String[] args) {
return new SpringApplication(primarySources).run(args);
}org.springframework.boot.SpringApplication#SpringApplication(org.springframework.core.io.ResourceLoader, java.lang.Class<?>...)
public SpringApplication(ResourceLoader resourceLoader, Class<?>... primarySources) {
this.resourceLoader = resourceLoader;
Assert.notNull(primarySources, "PrimarySources must not be null");
this.primarySources = new LinkedHashSet<>(Arrays.asList(primarySources));
this.webApplicationType = WebApplicationType.deduceFromClasspath();
// Load all classes that implement the ApplicationContextInitializer interface
setInitializers((Collection) getSpringFactoriesInstances(ApplicationContextInitializer.class));
// Load all classes that implement the ApplicationListener interface
setListeners((Collection) getSpringFactoriesInstances(ApplicationListener.class));
this.mainApplicationClass = deduceMainApplicationClass();
}org.springframework.boot.SpringApplication#run(java.lang.String...)
public ConfigurableApplicationContext run(String... args) {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
ConfigurableApplicationContext context = null;
Collection<SpringBootExceptionReporter> exceptionReporters = new ArrayList<>();
configureHeadlessProperty();
SpringApplicationRunListeners listeners = getRunListeners(args);
listeners.starting();
try {
ApplicationArguments applicationArguments = new DefaultApplicationArguments(args);
ConfigurableEnvironment environment = prepareEnvironment(listeners, applicationArguments);
configureIgnoreBeanInfo(environment);
Banner printedBanner = printBanner(environment);
context = createApplicationContext();
exceptionReporters = getSpringFactoriesInstances(SpringBootExceptionReporter.class,
new Class[] { ConfigurableApplicationContext.class }, context);
prepareContext(context, environment, listeners, applicationArguments, printedBanner);
refreshContext(context);
afterRefresh(context, applicationArguments);
stopWatch.stop();
if (this.logStartupInfo) {
new StartupInfoLogger(this.mainApplicationClass).logStarted(getApplicationLog(), stopWatch);
}
listeners.started(context);
callRunners(context, applicationArguments);
}
catch (Throwable ex) {
handleRunFailure(context, ex, exceptionReporters, listeners);
throw new IllegalStateException(ex);
}
try {
listeners.running(context);
}
catch (Throwable ex) {
handleRunFailure(context, ex, exceptionReporters, null);
throw new IllegalStateException(ex);
}
return context;
}In the run method, we focus on the prepareContext method. There is an applyInitializers method in this method
org.springframework.boot.SpringApplication#applyInitializers
protected void applyInitializers(ConfigurableApplicationContext context) {
// During initialization, load all classes that implement the ApplicationContextInitializer interface and execute the initialize method
for (ApplicationContextInitializer initializer : getInitializers()) {
Class<?> requiredType = GenericTypeResolver.resolveTypeArgument(initializer.getClass(),
ApplicationContextInitializer.class);
Assert.isInstanceOf(requiredType, context, "Unable to call initializer.");
initializer.initialize(context);
}
}In the process of starting the service, you can see from the console that the propertysourcebootstrap configuration class reads the corresponding configuration of Nacos, and this class also implements the ApplicationContextInitializer interface. Let's take a look at its initialize method.
org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration#initialize
public void initialize(ConfigurableApplicationContext applicationContext) {
List<PropertySource<?>> composite = new ArrayList<>();
AnnotationAwareOrderComparator.sort(this.propertySourceLocators);
boolean empty = true;
ConfigurableEnvironment environment = applicationContext.getEnvironment();
// Traverse all implementation classes that implement the PropertySourceLocator interface (under the dependency of using Nacos as the configuration center, its implementation class is only NacosPropertySourceLocator)
for (PropertySourceLocator locator : this.propertySourceLocators) {
// This is to call the locate method of the implementation class,
Collection<PropertySource<?>> source = locator.locateCollection(environment);
if (source == null || source.size() == 0) {
continue;
}
List<PropertySource<?>> sourceList = new ArrayList<>();
for (PropertySource<?> p : source) {
sourceList.add(new BootstrapPropertySource<>(p));
}
logger.info("Located property source: " + sourceList);
composite.addAll(sourceList);
empty = false;
}
if (!empty) {
MutablePropertySources propertySources = environment.getPropertySources();
String logConfig = environment.resolvePlaceholders("${logging.config:}");
LogFile logFile = LogFile.get(environment);
for (PropertySource<?> p : environment.getPropertySources()) {
if (p.getName().startsWith(BOOTSTRAP_PROPERTY_SOURCE_NAME)) {
propertySources.remove(p.getName());
}
}
insertPropertySources(propertySources, composite);
reinitializeLoggingSystem(environment, logConfig, logFile);
setLogLevels(applicationContext, environment);
handleIncludedProfiles(environment);
}
}com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#locate
public PropertySource<?> locate(Environment env) {
nacosConfigProperties.setEnvironment(env);
// Create NacosConfigService through reflection
ConfigService configService = nacosConfigManager.getConfigService();
if (null == configService) {
log.warn("no instance of config service found, can't load config from nacos");
return null;
}
long timeout = nacosConfigProperties.getTimeout();
nacosPropertySourceBuilder = new NacosPropertySourceBuilder(configService,
timeout);
// Get the value of name
String name = nacosConfigProperties.getName();
// Get the value of prefix
String dataIdPrefix = nacosConfigProperties.getPrefix();
if (StringUtils.isEmpty(dataIdPrefix)) {
// If prefix is empty, dataIdPrefix is the value of name
dataIdPrefix = name;
}
if (StringUtils.isEmpty(dataIdPrefix)) {
// If it is judged to be empty again, spring. Com is directly obtained application. Value of name
dataIdPrefix = env.getProperty("spring.application.name");
}
CompositePropertySource composite = new CompositePropertySource(
NACOS_PROPERTY_SOURCE_NAME);
// Load shared configuration
loadSharedConfiguration(composite);
// Load extended configuration
loadExtConfiguration(composite);
// Load current application configuration
loadApplicationConfiguration(composite, dataIdPrefix, nacosConfigProperties, env);
return composite;
}com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadSharedConfiguration
private void loadSharedConfiguration(
CompositePropertySource compositePropertySource) {
List<NacosConfigProperties.Config> sharedConfigs = nacosConfigProperties
.getSharedConfigs();
// Load shared configurations (i.e. configure shared configurations)
if (!CollectionUtils.isEmpty(sharedConfigs)) {
checkConfiguration(sharedConfigs, "shared-configs");
loadNacosConfiguration(compositePropertySource, sharedConfigs);
}
}com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadExtConfiguration
private void loadExtConfiguration(CompositePropertySource compositePropertySource) {
List<NacosConfigProperties.Config> extConfigs = nacosConfigProperties
.getExtensionConfigs();
// Load extension configuration (i.e. configure extension configurations)
if (!CollectionUtils.isEmpty(extConfigs)) {
checkConfiguration(extConfigs, "extension-configs");
loadNacosConfiguration(compositePropertySource, extConfigs);
}
}com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadApplicationConfiguration
private void loadApplicationConfiguration(
CompositePropertySource compositePropertySource, String dataIdPrefix,
NacosConfigProperties properties, Environment environment) {
// Get the file suffix, i.e. file extension (default properties)
String fileExtension = properties.getFileExtension();
// Get the group name, i.e. group (DEFAULT_GROUP)
String nacosGroup = properties.getGroup();
// load directly once by default
// Load dataIdPrefix as the configuration of dataId
loadNacosDataIfPresent(compositePropertySource, dataIdPrefix, nacosGroup,
fileExtension, true);
// load with suffix, which have a higher priority than the default
// Load dataIdPrefix + fileExtension as the configuration of dataId
loadNacosDataIfPresent(compositePropertySource,
dataIdPrefix + DOT + fileExtension, nacosGroup, fileExtension, true);
// Loaded with profile, which have a higher priority than the suffix
for (String profile : environment.getActiveProfiles()) {
String dataId = dataIdPrefix + SEP1 + profile + DOT + fileExtension;
// Load dataIdPrefix + profile + fileExtension as the configuration of dataId
loadNacosDataIfPresent(compositePropertySource, dataId, nacosGroup,
fileExtension, true);
}
}It can be seen that the order of configuration loading is shared configuration - > extension configuration - > application configuration. When loading application configuration, the specific priority order is prefix + environment + suffix > prefix + suffix > prefix (Note: the prefix is dataIdPrefix, the environment is profile, and the suffix is fileExtension).
The three loading methods are loadNacosDataIfPresent
com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosDataIfPresent
private void loadNacosDataIfPresent(final CompositePropertySource composite,
final String dataId, final String group, String fileExtension,
boolean isRefreshable) {
if (null == dataId || dataId.trim().length() < 1) {
return;
}
if (null == group || group.trim().length() < 1) {
return;
}
NacosPropertySource propertySource = this.loadNacosPropertySource(dataId, group,
fileExtension, isRefreshable);
this.addFirstPropertySource(composite, propertySource, false);
}com.alibaba.cloud.nacos.client.NacosPropertySourceLocator#loadNacosPropertySource
private NacosPropertySource loadNacosPropertySource(final String dataId,
final String group, String fileExtension, boolean isRefreshable) {
if (NacosContextRefresher.getRefreshCount() != 0) {
if (!isRefreshable) {
// No dynamic refresh is required, and it is obtained directly from the local cache
return NacosPropertySourceRepository.getNacosPropertySource(dataId,
group);
}
}
// Get from remote
return nacosPropertySourceBuilder.build(dataId, group, fileExtension,
isRefreshable);
}The process of obtaining the local cache is to obtain the dataid and group from the Map as the key. Here we focus on the method of obtaining from the remote.
com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder#build
NacosPropertySource build(String dataId, String group, String fileExtension,
boolean isRefreshable) {
// load configuration
Map<String, Object> p = loadNacosData(dataId, group, fileExtension);
NacosPropertySource nacosPropertySource = new NacosPropertySource(group, dataId,
p, new Date(), isRefreshable);
// Put into local cache
NacosPropertySourceRepository.collectNacosPropertySource(nacosPropertySource);
return nacosPropertySource;
}com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder#loadNacosData
private Map<String, Object> loadNacosData(String dataId, String group,
String fileExtension) {
String data = null;
try {
// The specific execution method is the getConfigInner method of the NacosConfigService class
data = configService.getConfig(dataId, group, timeout);
if (StringUtils.isEmpty(data)) {
log.warn(
"Ignore the empty nacos configuration and get it based on dataId[{}] & group[{}]",
dataId, group);
return EMPTY_MAP;
}
if (log.isDebugEnabled()) {
log.debug(String.format(
"Loading nacos data, dataId: '%s', group: '%s', data: %s", dataId,
group, data));
}
Map<String, Object> dataMap = NacosDataParserHandler.getInstance()
.parseNacosData(data, fileExtension);
return dataMap == null ? EMPTY_MAP : dataMap;
}
catch (NacosException e) {
log.error("get data from Nacos error,dataId:{}, ", dataId, e);
}
catch (Exception e) {
log.error("parse data from Nacos error,dataId:{},data:{},", dataId, data, e);
}
return EMPTY_MAP;
}com.alibaba.nacos.client.config.NacosConfigService#getConfigInner
private String getConfigInner(String tenant, String dataId, String group, long timeoutMs) throws NacosException {
group = null2defaultGroup(group);
ParamUtils.checkKeyParam(dataId, group);
ConfigResponse cr = new ConfigResponse();
cr.setDataId(dataId);
cr.setTenant(tenant);
cr.setGroup(group);
// Give priority to local configuration and try to get (/ data / config data) from the specified directory. By default, the path is ${user.home}/nacos/config/
String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
if (content != null) {
LOGGER.warn("[{}] [get-config] get failover ok, dataId={}, group={}, tenant={}, config={}", agent.getName(),
dataId, group, tenant, ContentUtils.truncateContent(content));
cr.setContent(content);
configFilterChainManager.doFilter(null, cr);
content = cr.getContent();
return content;
}
try {
// If the local configuration is not obtained, call the server to obtain it (/ v1/cs/configs)
String[] ct = worker.getServerConfig(dataId, group, tenant, timeoutMs);
cr.setContent(ct[0]);
configFilterChainManager.doFilter(null, cr);
content = cr.getContent();
return content;
} catch (NacosException ioe) {
if (NacosException.NO_RIGHT == ioe.getErrCode()) {
throw ioe;
}
LOGGER.warn("[{}] [get-config] get from server error, dataId={}, group={}, tenant={}, msg={}",
agent.getName(), dataId, group, tenant, ioe.toString());
}
LOGGER.warn("[{}] [get-config] get snapshot ok, dataId={}, group={}, tenant={}, config={}", agent.getName(),
dataId, group, tenant, ContentUtils.truncateContent(content));
// If it is not obtained from the server, it is directly in the form of local snapshot (/ snapshot)
content = LocalConfigInfoProcessor.getSnapshot(agent.getName(), dataId, group, tenant);
cr.setContent(content);
configFilterChainManager.doFilter(null, cr);
content = cr.getContent();
return content;
}The above is a process of loading configuration when the client starts, but how does the client refresh dynamically when the configuration information changes?
Configure dynamic refresh process
Client sends long polling request
When the client starts loading the configuration, you can see it on COM alibaba. cloud. nacos. client. In the method of nacospropertysourcelocator #locate, NacosConfigService was first created by using reflection. Here we take a look at its constructor
com.alibaba.nacos.client.config.NacosConfigService#NacosConfigService
public NacosConfigService(Properties properties) throws NacosException {
String encodeTmp = properties.getProperty(PropertyKeyConst.ENCODE);
if (StringUtils.isBlank(encodeTmp)) {
encode = Constants.ENCODE;
} else {
encode = encodeTmp.trim();
}
initNamespace(properties);
// MetricsHttpAgen is encapsulated by decorator mode. The real processing is ServerHttpAgent, which uses a thread pool with 1 core threads to call the login interface of the server every 5 seconds (/ v1/auth/users/login)
agent = new MetricsHttpAgent(new ServerHttpAgent(properties));
agent.start();
// Create a ClientWorker object
worker = new ClientWorker(agent, configFilterChainManager, properties);
}Here's how to create ClientWorker.
com.alibaba.nacos.client.config.impl.ClientWorker#ClientWorker
public ClientWorker(final HttpAgent agent, final ConfigFilterChainManager configFilterChainManager, final Properties properties) {
this.agent = agent;
this.configFilterChainManager = configFilterChainManager;
// Initialize the timeout parameter
init(properties);
// Initialize a thread pool with 1 core threads
executor = Executors.newScheduledThreadPool(1, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("com.alibaba.nacos.client.Worker." + agent.getName());
t.setDaemon(true);
return t;
}
});
// Initialize the thread pool for long polling. The number of core threads is runtime getRuntime(). availableProcessors()
executorService = Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors(), new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("com.alibaba.nacos.client.Worker.longPolling." + agent.getName());
t.setDaemon(true);
return t;
}
});
// The first delay is 1s, and the subsequent delay is 10s. Check the configuration information
executor.scheduleWithFixedDelay(new Runnable() {
@Override
public void run() {
try {
checkConfigInfo();
} catch (Throwable e) {
LOGGER.error("[" + agent.getName() + "] [sub-check] rotate check error", e);
}
}
}, 1L, 10L, TimeUnit.MILLISECONDS);
}com.alibaba.nacos.client.config.impl.ClientWorker#checkConfigInfo
public void checkConfigInfo() {
// Task fragmentation
int listenerSize = cacheMap.get().size();
// Round up to the number of batches to ensure that each task can be checked (paramutil. Getpertask configsize() defaults to 3000. Assuming there are 4000 configurations, it will be divided into two pieces)
int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
if (longingTaskCount > currentLongingTaskCount) {
for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) {
// Execute the task (LongPollingRunnable implements Runnable, and the specific execution method is the run method)
executorService.execute(new LongPollingRunnable(i));
}
currentLongingTaskCount = longingTaskCount;
}
}com.alibaba.nacos.client.config.impl.ClientWorker.LongPollingRunnable#run
public void run() {
List<CacheData> cacheDatas = new ArrayList<CacheData>();
List<String> inInitializingCacheList = new ArrayList<String>();
try {
// check failover config
for (CacheData cacheData : cacheMap.get().values()) {
// Only the configuration of the current slice is processed
if (cacheData.getTaskId() == taskId) {
cacheDatas.add(cacheData);
try {
// Check local configuration
checkLocalConfig(cacheData);
// Judge whether to use local configuration. If true, perform md5 comparison
if (cacheData.isUseLocalConfigInfo()) {
cacheData.checkListenerMd5();
}
} catch (Exception e) {
LOGGER.error("get local config info error", e);
}
}
}
// Check whether the configuration under the current partition has changed on the server, and return the changed key,
List<String> changedGroupKeys = checkUpdateDataIds(cacheDatas, inInitializingCacheList);
LOGGER.info("get changedGroupKeys:" + changedGroupKeys);
// Traverse the changed key, request the server interface to obtain the latest configuration according to the key, and update it to the cache
for (String groupKey : changedGroupKeys) {
String[] key = GroupKey.parseKey(groupKey);
String dataId = key[0];
String group = key[1];
String tenant = null;
if (key.length == 3) {
tenant = key[2];
}
try {
String[] ct = getServerConfig(dataId, group, tenant, 3000L);
CacheData cache = cacheMap.get().get(GroupKey.getKeyTenant(dataId, group, tenant));
cache.setContent(ct[0]);
if (null != ct[1]) {
cache.setType(ct[1]);
}
LOGGER.info("[{}] [data-received] dataId={}, group={}, tenant={}, md5={}, content={}, type={}",
agent.getName(), dataId, group, tenant, cache.getMd5(),
ContentUtils.truncateContent(ct[0]), ct[1]);
} catch (NacosException ioe) {
String message = String.format(
"[%s] [get-update] get changed config exception. dataId=%s, group=%s, tenant=%s",
agent.getName(), dataId, group, tenant);
LOGGER.error(message, ioe);
}
}
// Traverse the configuration under the partition. If the isInitializing of the configuration is false (that is, it does not appear in the cacheMap for the first time) or the configuration exists in the ininitializing cachelist, compare md5 and set isInitializing to false
for (CacheData cacheData : cacheDatas) {
if (!cacheData.isInitializing() || inInitializingCacheList
.contains(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant))) {
cacheData.checkListenerMd5();
cacheData.setInitializing(false);
}
}
// Clear first cache configuration list
inInitializingCacheList.clear();
// Continue with the task
executorService.execute(this);
} catch (Throwable e) {
// If the rotation training task is abnormal, the next execution time of the task will be punished
// If an exception occurs, the next round of task execution will be extended
LOGGER.error("longPolling error : ", e);
executorService.schedule(this, taskPenaltyTime, TimeUnit.MILLISECONDS);
}
}com.alibaba.nacos.client.config.impl.ClientWorker#checkLocalConfig
private void checkLocalConfig(CacheData cacheData) {
final String dataId = cacheData.dataId;
final String group = cacheData.group;
final String tenant = cacheData.tenant;
// Locally configured files
File path = LocalConfigInfoProcessor.getFailoverFile(agent.getName(), dataId, group, tenant);
// If the local configuration is not used and the local configuration file exists, the local configuration is used and isUseLocalConfig is set to true (that is, the local configuration is used)
if (!cacheData.isUseLocalConfigInfo() && path.exists()) {
String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
String md5 = MD5.getInstance().getMD5String(content);
cacheData.setUseLocalConfigInfo(true);
cacheData.setLocalConfigInfoVersion(path.lastModified());
cacheData.setContent(content);
LOGGER.warn("[{}] [failover-change] failover file created. dataId={}, group={}, tenant={}, md5={}, content={}",
agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
return;
}
// If the local configuration is used and the local configuration file does not exist, directly set isUseLocalConfig to false (do not use the local configuration)
if (cacheData.isUseLocalConfigInfo() && !path.exists()) {
cacheData.setUseLocalConfigInfo(false);
LOGGER.warn("[{}] [failover-change] failover file deleted. dataId={}, group={}, tenant={}", agent.getName(),
dataId, group, tenant);
return;
}
// When the local configuration is used and the locally configured file exists and the version in the current memory is inconsistent with that of the local file, use the configuration of the local file and set the version in memory to the version of the local file
if (cacheData.isUseLocalConfigInfo() && path.exists()
&& cacheData.getLocalConfigInfoVersion() != path.lastModified()) {
String content = LocalConfigInfoProcessor.getFailover(agent.getName(), dataId, group, tenant);
String md5 = MD5.getInstance().getMD5String(content);
cacheData.setUseLocalConfigInfo(true);
cacheData.setLocalConfigInfoVersion(path.lastModified());
cacheData.setContent(content);
LOGGER.warn("[{}] [failover-change] failover file changed. dataId={}, group={}, tenant={}, md5={}, content={}",
agent.getName(), dataId, group, tenant, md5, ContentUtils.truncateContent(content));
}
}com.alibaba.nacos.client.config.impl.ClientWorker#checkUpdateDataIds
List<String> checkUpdateDataIds(List<CacheData> cacheDatas, List<String> inInitializingCacheList) throws IOException {
StringBuilder sb = new StringBuilder();
// Traverse the configuration information in the cache under the partition and splice the data that does not use the local configuration
for (CacheData cacheData : cacheDatas) {
if (!cacheData.isUseLocalConfigInfo()) {
sb.append(cacheData.dataId).append(WORD_SEPARATOR);
sb.append(cacheData.group).append(WORD_SEPARATOR);
if (StringUtils.isBlank(cacheData.tenant)) {
sb.append(cacheData.getMd5()).append(LINE_SEPARATOR);
} else {
sb.append(cacheData.getMd5()).append(WORD_SEPARATOR);
sb.append(cacheData.getTenant()).append(LINE_SEPARATOR);
}
if (cacheData.isInitializing()) {
// cacheData appears in the cacheMap for the first time and is updated by check for the first time
inInitializingCacheList
.add(GroupKey.getKeyTenant(cacheData.dataId, cacheData.group, cacheData.tenant));
}
}
}
boolean isInitializingCacheList = !inInitializingCacheList.isEmpty();
// Check data that does not use local configuration
return checkUpdateConfigStr(sb.toString(), isInitializingCacheList);
}com.alibaba.nacos.client.config.impl.ClientWorker#checkUpdateConfigStr
List<String> checkUpdateConfigStr(String probeUpdateString, boolean isInitializingCacheList) throws IOException {
List<String> params = new ArrayList<String>(2);
params.add(Constants.PROBE_MODIFY_REQUEST);
params.add(probeUpdateString);
List<String> headers = new ArrayList<String>(2);
headers.add("Long-Pulling-Timeout");
headers.add("" + timeout);
// told server do not hang me up if new initializing cacheData added in
if (isInitializingCacheList) {
headers.add("Long-Pulling-Timeout-No-Hangup");
headers.add("true");
}
if (StringUtils.isBlank(probeUpdateString)) {
return Collections.emptyList();
}
try {
// In order to prevent the server from handling the delay of the client's long task,
// increase the client's read timeout to avoid this problem.
// Initiate a long polling request. The default timeout is 45S. The request path is: / nacos/v1/ns/configs/listener
long readTimeoutMs = timeout + (long) Math.round(timeout >> 1);
HttpResult result = agent.httpPost(Constants.CONFIG_CONTROLLER_PATH + "/listener", headers, params,
agent.getEncode(), readTimeoutMs);
if (HttpURLConnection.HTTP_OK == result.code) {
setHealthServer(true);
// If successful, the response content will be parsed and the changed groupKey will be returned
return parseUpdateDataIdResponse(result.content);
} else {
setHealthServer(false);
LOGGER.error("[{}] [check-update] get changed dataId error, code: {}", agent.getName(), result.code);
}
} catch (IOException e) {
setHealthServer(false);
LOGGER.error("[" + agent.getName() + "] [check-update] get changed dataId exception", e);
throw e;
}
return Collections.emptyList();
}The server handles long polling requests
com.alibaba.nacos.config.server.controller.ConfigController#listener
public void listener(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
request.setAttribute("org.apache.catalina.ASYNC_SUPPORTED", true);
// String spliced without local configuration information
String probeModify = request.getParameter("Listening-Configs");
if (StringUtils.isBlank(probeModify)) {
throw new IllegalArgumentException("invalid probeModify");
}
log.info("listen config id:" + probeModify);
probeModify = URLDecoder.decode(probeModify, Constants.ENCODE);
Map<String, String> clientMd5Map;
try {
// Parse string
clientMd5Map = MD5Util.getClientMd5Map(probeModify);
} catch (Throwable e) {
throw new IllegalArgumentException("invalid probeModify");
}
log.info("listen config id 2:" + probeModify);
// do long-polling
// Execute long polling
inner.doPollingConfig(request, response, clientMd5Map, probeModify.length());
}com.alibaba.nacos.config.server.controller.ConfigServletInner#doPollingConfig
public String doPollingConfig(HttpServletRequest request, HttpServletResponse response,
Map<String, String> clientMd5Map, int probeRequestSize)
throws IOException {
// Judge whether there is long pulling timeout according to the request header. If so, it is long polling
if (LongPollingService.isSupportLongPolling(request)) {
// Long polling, blocking the return of requests
longPollingService.addLongPollingClient(request, response, clientMd5Map, probeRequestSize);
return HttpServletResponse.SC_OK + "";
}
// Short polling: traverse the configured md5 and return the changed groupKey
List<String> changedGroups = MD5Util.compareMd5(request, response, clientMd5Map);
// Compatible short polling result
String oldResult = MD5Util.compareMd5OldResult(changedGroups);
String newResult = MD5Util.compareMd5ResultString(changedGroups);
String version = request.getHeader(Constants.CLIENT_VERSION_HEADER);
if (version == null) {
version = "2.0.0";
}
int versionNum = Protocol.getVersionNumber(version);
/**
* 2.0.4 Before version, the return value is placed in the header
*/
if (versionNum < START_LONGPOLLING_VERSION_NUM) {
response.addHeader(Constants.PROBE_MODIFY_RESPONSE, oldResult);
response.addHeader(Constants.PROBE_MODIFY_RESPONSE_NEW, newResult);
} else {
request.setAttribute("content", newResult);
}
Loggers.AUTH.info("new content:" + newResult);
// disable cache
response.setHeader("Pragma", "no-cache");
response.setDateHeader("Expires", 0);
response.setHeader("Cache-Control", "no-cache,no-store");
response.setStatus(HttpServletResponse.SC_OK);
return HttpServletResponse.SC_OK + "";
}The method of long polling plus short polling is implemented here. For short polling directly called by ordinary requests, the results are directly returned in the content and directly parsed by the client. See the long polling method below
com.alibaba.nacos.config.server.service.LongPollingService#addLongPollingClient
public void addLongPollingClient(HttpServletRequest req, HttpServletResponse rsp, Map<String, String> clientMd5Map,
int probeRequestSize) {
String str = req.getHeader(LongPollingService.LONG_POLLING_HEADER);
String noHangUpFlag = req.getHeader(LongPollingService.LONG_POLLING_NO_HANG_UP_HEADER);
String appName = req.getHeader(RequestUtil.CLIENT_APPNAME_HEADER);
String tag = req.getHeader("Vipserver-Tag");
int delayTime = SwitchService.getSwitchInteger(SwitchService.FIXED_DELAY_TIME, 500);
/**
* Return the response 500ms in advance to avoid client timeout @ Qiaoyi dingqy 2013.10. 22 add delay time for LoadBalance
*/
long timeout = Math.max(10000, Long.parseLong(str) - delayTime);
if (isFixedPolling()) {
timeout = Math.max(10000, getFixedPollingInterval());
// do nothing but set fix polling timeout
} else {
long start = System.currentTimeMillis();
// By comparing md5, the changed groupKey is returned
List<String> changedGroups = MD5Util.compareMd5(req, rsp, clientMd5Map);
// If the key list is greater than 0, the data will be returned directly to end the long polling
if (changedGroups.size() > 0) {
generateResponse(req, rsp, changedGroups);
LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}",
System.currentTimeMillis() - start, "instant", RequestUtil.getRemoteIp(req), "polling",
clientMd5Map.size(), probeRequestSize, changedGroups.size());
return;
} else if (noHangUpFlag != null && noHangUpFlag.equalsIgnoreCase(TRUE_STR)) {
LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}", System.currentTimeMillis() - start, "nohangup",
RequestUtil.getRemoteIp(req), "polling", clientMd5Map.size(), probeRequestSize,
changedGroups.size());
return;
}
}
String ip = RequestUtil.getRemoteIp(req);
// It must be called by the HTTP thread, or the container will send a response immediately after leaving
final AsyncContext asyncContext = req.startAsync();
// AsyncContext. The timeout of settimeout() is not allowed, so you can only control it yourself
asyncContext.setTimeout(0L);
// Start a new long polling thread
scheduler.execute(
new ClientLongPolling(asyncContext, clientMd5Map, ip, probeRequestSize, timeout, appName, tag));
}The main logic here is as follows:
1. Read the corresponding data from the request header and calculate the timeout, which is 30000-500ms by default to ensure that the response can be given to the client before the timeout
2. Judge whether isFixedPolling is true, and if it is true, recalculate the timeout; Go to point 3 for false
3. If isFixedPolling is false, the configuration is compared through md5. If there is a changed configuration, it is directly returned to the client to end the long polling; No change configuration, go to point 4
4. If the configuration is not modified, it is judged that the long pulling timeout no hangup of the request header is not empty and true, and it is returned directly
5. If none of the above logic returns, AsyncContext asynchronous processing is created through the request, and a new long polling task is created through the thread pool
com.alibaba.nacos.config.server.service.LongPollingService.ClientLongPolling#run
public void run() {
asyncTimeoutFuture = scheduler.schedule(new Runnable() {
@Override
public void run() {
try {
getRetainIps().put(ClientLongPolling.this.ip, System.currentTimeMillis());
// Remove the current long polling instance from the queue
allSubs.remove(ClientLongPolling.this);
if (isFixedPolling()) {
// If it is a fixed long polling, find the updated groupKey through md5 comparison, and return
LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}",
(System.currentTimeMillis() - createTime),
"fix", RequestUtil.getRemoteIp((HttpServletRequest)asyncContext.getRequest()),
"polling",
clientMd5Map.size(), probeRequestSize);
List<String> changedGroups = MD5Util.compareMd5(
(HttpServletRequest)asyncContext.getRequest(),
(HttpServletResponse)asyncContext.getResponse(), clientMd5Map);
if (changedGroups.size() > 0) {
sendResponse(changedGroups);
} else {
sendResponse(null);
}
} else {
LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}",
(System.currentTimeMillis() - createTime),
"timeout", RequestUtil.getRemoteIp((HttpServletRequest)asyncContext.getRequest()),
"polling",
clientMd5Map.size(), probeRequestSize);
sendResponse(null);
}
} catch (Throwable t) {
LogUtil.defaultLog.error("long polling error:" + t.getMessage(), t.getCause());
}
}
}, timeoutTime, TimeUnit.MILLISECONDS);
// Add the long polling instance to the queue
allSubs.add(this);
}It can be seen from this code that this long polling task is a delayed task. If the configuration changes during the delayed period, how can the client dynamically perceive it? My guess is that there will be event monitoring.
// LongPollingService inherits AbstractEventListener and adds listeners by rewriting
public class LongPollingService extends AbstractEventListener {
public List<Class<? extends Event>> interest() {
List<Class<? extends Event>> eventTypes = new ArrayList<Class<? extends Event>>();
// Added LocalDataChangeEvent listener
eventTypes.add(LocalDataChangeEvent.class);
return eventTypes;
}
public void onEvent(Event event) {
if (isFixedPolling()) {
// ignore
} else {
if (event instanceof LocalDataChangeEvent) {
LocalDataChangeEvent evt = (LocalDataChangeEvent)event;
// Execute the DataChangeTask task through the thread pool
scheduler.execute(new DataChangeTask(evt.groupKey, evt.isBeta, evt.betaIps));
}
}
}
}com.alibaba.nacos.config.server.service.LongPollingService.DataChangeTask#run
public void run() {
try {
ConfigService.getContentBetaMd5(groupKey);
// Traverse the long polling instance queue
for (Iterator<ClientLongPolling> iter = allSubs.iterator(); iter.hasNext(); ) {
ClientLongPolling clientSub = iter.next();
// Long polling instance found matching
if (clientSub.clientMd5Map.containsKey(groupKey)) {
// If the beta is released and is not in the beta list, skip it directly
if (isBeta && !betaIps.contains(clientSub.ip)) {
continue;
}
// If the tag is published and not in the tag list, skip it directly
if (StringUtils.isNotBlank(tag) && !tag.equals(clientSub.tag)) {
continue;
}
getRetainIps().put(clientSub.ip, System.currentTimeMillis());
// Remove the long polling instance from the allSubs queue
iter.remove();
LogUtil.clientLog.info("{}|{}|{}|{}|{}|{}|{}",
(System.currentTimeMillis() - changeTime),
"in-advance",
RequestUtil.getRemoteIp((HttpServletRequest)clientSub.asyncContext.getRequest()),
"polling",
clientSub.clientMd5Map.size(), clientSub.probeRequestSize, groupKey);
// Send the return information (judge whether the delayed scheduling task is cancelled, if not, cancel it first, and then return the data)
clientSub.sendResponse(Arrays.asList(groupKey));
}
}
} catch (Throwable t) {
LogUtil.defaultLog.error("data change error:" + t.getMessage(), t.getCause());
}
}On the server side, you can see a series of isFixedPolling judgments. This means fixed long polling (client pull mode)
summary
By viewing the source code of the entire configuration center, it can be seen that Nacos configuration center realizes the pull of the client and the push of the server. The specific steps are as follows:
1. Client initiated request
2. After receiving the request, the server compares whether it has changed through md5; If there is a change, directly return the data and return to step 1. If there is no change, directly suspend the request
3. If the data has not changed during this period, the request is ended; If there is a change, the server will poll and listen to the configuration item request instance and directly return data
It is worth noting that the server will not return content here, but only groupKey(dataId+groupId)
