Cross domain is both familiar and unfamiliar to back-end engineers.
In the past two months, I participated in the incubation of an educational product as an architect and had an unforgettable cross domain trip.
Writing this article, I want to share my experience and thinking in cross domain knowledge, hoping to inspire you.
1. Cross domain
The products have multiple ends: Institutional end, bureau end, parent end, etc. Each end has separate domain names, some on PC, some through WeChat official account, and some H5 after scanning.
The interface domain name called by the access layer uses API training. COM, an independent domain name, configures request forwarding through Nginx.
Generally, cross domain refers to CORS.
CORS is a W3C standard. Its full name is "cross origin resource sharing". It requires the browser and server to support it at the same time, allowing the browser to send XMLHttpRequest requests to cross source servers, so as to overcome the limitation that AJAX can only be used in the same source.
So how to define homology? Let's first look at the address of the next typical website:
Homology means that the protocol, domain name and port number are identical.
The following table shows the and URLs http://www.training.com/dir/page.html An example of comparing the source of:
When the user accesses the application through the browser( http://admin.training.com )The domain name of the calling interface is a non homologous domain name( http://api.training.com ), this is an obvious cross domain scenario.
2 CORS details
The cross domain resource sharing standard adds a set of HTTP header fields to allow the server to declare which source stations have access to which resources through the browser.
The specification requires that for HTTP request methods that may have side effects on server data (especially HTTP requests other than GET or POST requests with some MIME types), the browser must first use the OPTIONS method to initiate a preflight request to know whether the server allows the cross domain request.
After the server confirms that it is allowed, it initiates the actual HTTP request. In the return of the pre check request, the server can also notify the client whether it needs to carry identity credentials (including Cookies and HTTP authentication related data).
2.1 simple request
When the request meets the following conditions at the same time, CORS authentication mechanism will use simple request, otherwise CORS authentication mechanism will use pre check request.
- Use one of GET, POST and HEAD methods;
Only the following security header fields are used, and other header fields shall not be set manually;
- Accept
- Accept-Language
- Content-Language
- Content type is limited to one of three types: text/plain, multipart / form data, application/x-www-form-urlencoded:
- HTML header field: DPR, Download, save data, viewport WIdth, WIdth
- Any XMLHttpRequestUpload object in the request does not register any event listener; XMLHttpRequestUpload object can use XMLHttpRequest Upload attribute access;
- ReadableStream object is not used in the request.
In the simple request mode, the browser directly sends a cross domain request and carries the Origin header in the request header, indicating that this is a cross domain request. After receiving the request, the server will return the verification result through the access control allow Origin and access control allow methods response headers according to its own cross domain rules.
The response carries the cross domain header access control allow Origin. Using Origin and access control allow Origin can complete the simplest access control. In this example, the access control allow Origin returned by the server: * indicates that the resource can be accessed by any foreign domain. If the server only allows from http://admin.training.com The contents of the header field are as follows:
Access-Control-Allow-Origin: http://admin.training.com
Now, except http://admin.training.com , no other foreign domain can access this resource.
2.2 pre inspection request
When the browser finds that the request sent by the page is not a simple request, it will not immediately execute the corresponding request code, but will trigger the pre request mode. In the pre request mode, a preflight request will be sent first. The preflight request is an OPTION request, which is used to ask the server to be accessed across domains whether the page under the current domain name is allowed to send cross domain requests. The real HTTP request can only be sent after the cross domain authorization of the server is obtained.
The OPTIONS request header contains the following headers:
After receiving the OPTIONS request, the server sets the header to communicate with the browser to determine whether to allow the request.
If the preflight request passes the verification, the browser will send a real cross domain request.
3 back end configuration
I have tried two methods for backend configuration. After two months of testing, they can run very stably.
- Nginx configuration recommended by MND;
- SpringBoot comes with CorsFilter configuration.
▍ Nginx configuration recommended by MND
Nginx configuration is equivalent to configuration at the request forwarding layer.
location / {
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain; charset=utf-8';
add_header 'Content-Length' 0;
return 204;
}
if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
}
}When configuring the access control allow headers property, there are a large number of custom headers because they contain signatures and token s. For simplicity and convenience, I configure access control allow headers as *.
There are no exceptions under Chrome and firefox, but the following errors are reported under IE11:
The request header content type does not exist in the access control allow headers list.
Originally, IE11 required that the values of access control allow headers returned by the pre check request must be separated by commas.
▍ SpringBoot comes with CorsFilter
First, the following cross domain configurations are available in the basic framework by default.
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**")
.allowedOrigins("*")
.allowedMethods("POST", "GET", "PUT", "OPTIONS", "DELETE")
.allowCredentials(true)
.allowedHeaders("*")
.maxAge(3600);
}However, after the deployment is completed, the CORS exception is still reported:
From the nginx and tomcat logs, only one OPTION request is received. There is an interceptor ActionInterceptor in the springboot application, which obtains the token from the header, calls the user service to query the user information and puts it into the request. When the token data is not obtained, it will be returned to the front-end JSON format data.
But judging from the phenomenon, CorsMapping did not take effect.
Why? In fact, it is the concept of execution order. The following figure shows the execution sequence of filter, interceptor and controller.
DispatchServlet. The dodispatch () method is the core entry method of spring MVC.
// Determine handler for the current request.
mappedHandler = getHandler(processedRequest);
if (!mappedHandler.applyPreHandle(processedRequest, response)) {
return;
}
// Actually invoke the handler.
mv = ha.handle(processedRequest, response, mappedHandler.getHandler());So where is CorsMapping initialized? After debugging, it is located in AbstractHandlerMapping.
protected HandlerExecutionChain getCorsHandlerExecutionChain(HttpServletRequest request,
HandlerExecutionChain chain, CorsConfiguration config) {
if (CorsUtils.isPreFlightRequest(request)) {
HandlerInterceptor[] interceptors = chain.getInterceptors();
chain = new HandlerExecutionChain(new PreFlightHandler(config), interceptors);
}
else {
chain.addInterceptor(new CorsInterceptor(config));
}
return chain;
}There is pre check judgment in the code through preflighthandler Handled in handlerequest(), but after the normal business interceptor.
CorsFilter was finally selected for two reasons:
- The execution order priority of the filter is the highest;
- By debugging the source code of CorsFilter, it is found that the source code has many details.
private CorsConfiguration corsConfig() {
CorsConfiguration corsConfiguration = new CorsConfiguration();
corsConfiguration.addAllowedOrigin("*");
corsConfiguration.addAllowedHeader("*");
corsConfiguration.addAllowedMethod("*");
corsConfiguration.setAllowCredentials(true);
corsConfiguration.setMaxAge(3600L);
return corsConfiguration;
}
@Bean
public CorsFilter corsFilter() {
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
source.registerCorsConfiguration("/**", corsConfig());
return new CorsFilter(source);
}In the following code, when alloweheader is a wildcard *, CorsFilter will splice access control request headers with commas when setting access control allow headers, so as to avoid the problem of IE11 response headers.
public List<String> checkHeaders(@Nullable List<String> requestHeaders) {
if (requestHeaders == null) {
return null;
}
if (requestHeaders.isEmpty()) {
return Collections.emptyList();
}
if (ObjectUtils.isEmpty(this.allowedHeaders)) {
return null;
}
boolean allowAnyHeader = this.allowedHeaders.contains(ALL);
List<String> result = new ArrayList<>(requestHeaders.size());
for (String requestHeader : requestHeaders) {
if (StringUtils.hasText(requestHeader)) {
requestHeader = requestHeader.trim();
if (allowAnyHeader) {
result.add(requestHeader);
}
else {
for (String allowedHeader : this.allowedHeaders) {
if (requestHeader.equalsIgnoreCase(allowedHeader)) {
result.add(requestHeader);
break;
}
}
}
}
}
return (result.isEmpty() ? null : result);
}The execution effect of the browser is as follows:
4 preflight response code: 200 vs 204
After the back-end configuration was completed, the little partner in the team asked me, "brother Yong, is the response code returned by the pre inspection request 200 or 204?". This question really stopped me.
The pre check response code of our API gateway is 200, and the pre check response code of CorsFilter is also 200.
The sample pre check response codes given by MDN are all 204.
https://developer.mozilla.org...
I can only adopt Google's method, and I am surprised to find that the developers of the famous API gateway Kong have also discussed this issue.
The preflight response code recommended by MDN was 200, so Kong also synchronized with MDN to 200;
The page was updated since then. See its contents on Sept 30th, 2018:
Later, the MDN modified the response code 204, so the developers of Kong debated whether to synchronize with the MDN.
The core of the debate is: is there an urgent need. The 200 response code works well and seems to work normally forever. Replace it with 204, and it is uncertain whether there is a hidden problem.
- In the final analysis, framework developers still rely on the underlying implementation of the browser. On this issue, there is not enough authoritative data to support the framework developers, and all knowledge points are scattered in every corner of the network, filled with incomplete details and partial solutions, which make the framework developers very confused.
Finally, the pre check response code in Kong's source code is still 200, which is not synchronized with the MDN.
I carefully checked the major mainstream websites, and 95% of the pre inspection response code was 200. After more than two months of testing, Nginx configured the pre check response code 204, and there were no problems in the mainstream browsers chrome, Firefox and ie11.
Therefore, 200 works everywhere and 204 are also well supported in the current mainstream browsers.
5 chrome: non secure private network
I thought the cross domain problem was solved in this way. I didn't expect there was still an episode.
The product director needs to make a presentation to the customer, and I am responsible for fixing the presentation environment. Applying for a domain name, preparing alicloud server, application packaging and deployment went well.
However, when accessing the demonstration environment on the company's intranet, one page always reports CORS errors, and the error contents are similar to the following figure:
Cross domain error types are: < font color = "red" > insecureprivatenetwork < / font >.
This is totally different from the original cross domain error. I was flustered. Immediately Google, it turned out that this is a new feature after chrome was updated to 94. You can turn off this feature manually.
- Open tab page chrome://flags/#block-insecure-private-network-requests
- Set the block secure private network requests to Disabled, and then restart it. This is equivalent to disabling this function.
But this is a temporary cure, not a permanent cure. It's a little strange that when we don't access the demonstration environment on the company's intranet, the demonstration environment is completely normal, and the wrong pages can be accessed normally.
Looking carefully at the official documents, CORS-RFC1918 points out that the following three requests will be affected.
- Public network access to private network;
- Public network access to local equipment;
- Private network access to local devices.
In this way, I locate the problem on the wrong third-party interface address. Many products of the company rely on this interface service. When accessing the company's intranet, the domain name mapping address is similar: 172.16 xx. xx.
This ip is exactly the private network specified in rfc1918.
10.0.0.0 - 10.255.255.255 (10/8 prefix) 172.16.0.0 - 172.31.255.255 (172.16/12 prefix) 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
When the intranet accesses this page through Chrome, it will trigger non secure private network interception.
How to solve it? The official plan is divided into two steps:
- Private networks can only be accessed through Https;
- In the future, specific pre check headers will be added, such as access control request private network.
Of course, there are some temporary methods:
- Turn off the Chrome feature;
- Switch to other browsers, such as Firefox;
- Turn off the intranet and turn on the mobile phone hotspot;
- Modify the local host and bind the external ip.
Based on the official scheme, the production environment completely uses Https, so there is no such cross domain problem in the company's intranet access.
6 double disk
API gateway is very suitable for the architecture of current products. At the beginning of architecture design, many terminals of the system will call our API gateway. The API gateway can be deployed in SAAS and privatization. It has a separate domain name and provides a perfect signature algorithm. Considering the launch time node, the familiarity of team members with API gateways and the time cost of deploying multiple environments, I made some balance and compromise from the architecture level in order to deliver as soon as possible.
The interface domain name called by the access layer uses API training. COM, an independent domain name, configures request forwarding through Nginx. At the same time, I have unified the front-end and back-end protocols with the front-end Leader to keep consistent with our API gateway, so as to make preparations for the subsequent switching back to the API gateway.
API gateway can be used for authentication, current limiting, grayscale, etc., and CORS can be configured at the same time. The internal server does not need to pay special attention to cross domain.
At the same time, my mentality has also changed in the process of solving cross domain problems. From the initial contempt to the gradual sinking of heart, we can understand the principle of CORS step by step and distinguish the advantages and disadvantages of different solutions, and things will slowly go smoothly. I also observed: "some project teams have fed back Chrome non secure private network problems and given solutions. For technical managers, we must pay attention to the feedback problems in the project, do a good job in combing and analyzing, and sort out the plan. In this way, when similar problems arise, they will be orderly ".
7 write to the end
In 2017, I attended a technical speech by teacher Chen Hao, the left ear mouse, who told us a story.
The story is probably: "the company's software has an inexplicable BUG, and the user's fee is deducted, but there are often network problems when calling the third-party interface. The most powerful person of the company checked for a week and didn't solve it, and Mr. Chen Hao is watching TCP/IP detailed explanation." In this book, netstat, the connection status is CLOSE_WAIT means that the other party is disconnected. It is estimated that it is a problem with the other party's system. So he went to the other side to help them look at the code. Sure enough, there was a problem with the judgment conditions, resulting in the application directly disconnecting the link. It took less than two hours to solve the problem.
When I think of Mr. Chen Hao's story and look back on my cross domain journey, I deeply feel that details are demons, and solving problems may be in some inadvertent details.
If my article is helpful to you, please also like, read and forward it. Your support will encourage me to output higher quality articles. Thank you very much!
