hue uses http request to access the interface

Posted by ojsimon on Wed, 09 Feb 2022 18:08:13 +0100

There is a requirement in the development: after creating ldap users, they need to be automatically synchronized to hue After viewing the official documents, you can use http to request the interface of hue and synchronize ldap users

Record the pit of this interface call and the solution

1, hue's landing

The related interface requests of hue need to be completed in the same session, that is, the subsequent interface access can only be carried out after the login operation. Otherwise, 403 or login # require will be reported

It is very difficult to use csrf authentication for the login and interface access of hue

Steps of login: (1) request "hue/accounts/login /" and obtain csrfToken and sessionId

(2) For the second time, use the account, password, and the last saved csrfToken and sessionId to request "hue/accounts/login / to log in successfully, and save the csrfToken and sessionId

2, Interface requesting synchronization: useradmin/users/add_ldap_users and hue/useradmin/users/add_ldap_users completed the synchronization operation

Use the saved csrfToken, sessionId and other parameters: x-requested-with, referer and username_ Pattern, etc. complete the request

The java version code is attached below

public class HueRestUtil {
    private static final Logger logger = LoggerFactory.getLogger(HueUiConf.class);

    private String hueHost;
    private String hueUserName;
    private String huePassword;

    private String csrfToken;
    private String sessionId;
    private RestTemplate restTemplate = new RestTemplate();

    private HueRestUtil() {
    }

    public HueRestUtil(String hueHost, String hueUserName, String huePassword) {
        this.hueHost = hueHost;
        this.hueUserName = hueUserName;
        this.huePassword = huePassword;
        logInHue();
    }

    public String getHueHost() {
        return hueHost;
    }

    public String getHueUserName() {
        return hueUserName;
    }

    public String getHuePassword() {
        return huePassword;
    }

    public String getCsrfToken() {
        return csrfToken;
    }

    public String getSessionId() {
        return sessionId;
    }

   
    private void logInHue() {
        String url = String.join("", hueHost, "hue/accounts/login/");
        MultiValueMap<String, String> urlSuffix = new LinkedMultiValueMap<>();
        urlSuffix.add("fromModal", "true");
        URI uri = UriComponentsBuilder.fromHttpUrl(url).queryParams(urlSuffix).build().toUri();
        HttpHeaders headers = new HttpHeaders();
        try {
            // First login to obtain cookies, CSRF authentication information (get)
            ResponseEntity<String> exchange = restTemplate.exchange(uri, HttpMethod.GET, new HttpEntity<>(headers),
                    String.class);
            if (!exchange.getStatusCode().isError()) {
                for (Map.Entry<String, List<String>> entry : exchange.getHeaders().entrySet()) {
                    if (HttpHeaders.SET_COOKIE.equals(entry.getKey())) {
                        List<String> value = entry.getValue();
                        if (0 < value.size()) {
                            this.csrfToken = value.get(0).split(";")[0];
                            this.sessionId = value.get(1).split(";")[0];
                        }
                    }
                }
                // Set the parameter information related to the second login
                // Set body related information
                MultiValueMap<String, String> body = new LinkedMultiValueMap<>(2);
                body.add("username", hueUserName);
                body.add("password", huePassword);
                body.add("server", "LDAP");
                body.add("next", "/");
                body.add("csrfmiddlewaretoken", this.csrfToken.split("=")[1]);

                // Set header related information
                HttpHeaders httpHeaders = new HttpHeaders();
                httpHeaders.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
                httpHeaders.add(HttpHeaders.COOKIE, join(";", this.csrfToken, this.sessionId));
                httpHeaders.set("X-CSRFToken", this.csrfToken.split("=")[1]);
                httpHeaders.set("Referer", hueHost);
                URI loginUri = UriComponentsBuilder.fromHttpUrl(url).queryParams(urlSuffix).build().toUri();
                // Second landing
                ResponseEntity<String> loginResult = restTemplate.exchange(loginUri, HttpMethod.POST,
                        new HttpEntity<>(body, httpHeaders), String.class);
                if (loginResult.getStatusCode().isError()) {
                    logger.error("hue server error,please check config!code:{}", loginResult.getStatusCode().value());
                } else {
                    for (Map.Entry<String, List<String>> entry : loginResult.getHeaders().entrySet()) {
                        List<String> value = entry.getValue();
                        if (HttpHeaders.SET_COOKIE.equals(entry.getKey())) {
                            if (0 < value.size()) {
                                // Save the csrftoken and sessionid of the second login
                                this.csrfToken = value.get(0).split(";")[0];
                                this.sessionId = value.get(1).split(";")[0];
                            } else {
                                throw new RuntimeException("hue login,second check failed...");
                            }
                        }
                    }
                    logger.info("log in hue suss");
                }
            } else {
                logger.error("hue server error,please check config!code:{}", exchange.getStatusCode().value());
            }
        } catch (Exception e) {
            logger.error("Hue lonIn failed!", e);
        }
    }

   
    public Boolean syncLdapUser(String userName) throws UnsupportedEncodingException {

        String url = String.join("", hueHost, "useradmin/users/add_ldap_users");
        String refererUrl = String.join("", hueHost, "hue/useradmin/users/add_ldap_users");
        HttpHeaders httpHeaders = new HttpHeaders();
        httpHeaders.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
        httpHeaders.add(HttpHeaders.COOKIE, join(";", this.csrfToken, this.sessionId));
        if (Strings.isNotBlank(this.csrfToken) && 1 < this.csrfToken.split("=").length) {
            httpHeaders.set("X-CSRFToken", this.csrfToken.split("=")[1]);
        }
        httpHeaders.set("X-requested-with", "XMLHttpRequest");
        httpHeaders.set("Referer", refererUrl);
        httpHeaders.set("Accept", "application/json");
        httpHeaders.set("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");

        MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
        params.add("csrfmiddlewaretoken", URLEncoder.encode(this.csrfToken.split("=")[1], StandardCharsets.UTF_8.name()));
        params.add("is_embeddable", URLEncoder.encode("true", StandardCharsets.UTF_8.name()));
        params.add("username_pattern", URLEncoder.encode(userName, StandardCharsets.UTF_8.name()));
        params.add("ensure_home_directory", URLEncoder.encode("on", StandardCharsets.UTF_8.name()));
        ResponseEntity<String> stringResponseEntity = restTemplate.postForEntity(url,
                new HttpEntity<MultiValueMap>(params, httpHeaders), String.class);
        String responseStr = stringResponseEntity.getBody();
        JSONObject jsonObject = JSON.parseObject(responseStr);
        Object status = jsonObject.get("status");
        if (!"0".equals(status)) {
            logger.error("sync hue ldapUser failed response:{}", responseStr);
            return false;
        }
        return true;
    }

}

There are a lot of stepping holes this time: 1. First, use postman to request the parameters of the browser's debugging mode. Later, it was found that the request needs to be completed in the same session. First, log in
2. The login hue has not been successful. Later, I learned that it is necessary to add "server" and "LDAP" in the body when logging in

3. In the browser, the values of csrfmiddlewaretoken and X-CSRFToken are different. I've been struggling with the generation method of csrfmiddlewaretoken. Later, I found that these two are actually one thing. You can fill in the value of CSRFToken. I don't know why the browser is different

4. The debugging is also very poor. Many problem interfaces directly return 403 or login # require. They don't know where to deal with the error reporting problem

last

This time, only the interface call from ldap users to hue is synchronized. The other call methods are basically the same. The call parameters can be viewed in browser debugging mode

Topics: Hadoop cloudera