After using ElasticSearch, the query speed of the company's system is 50 times faster

Posted by Vebut on Thu, 17 Feb 2022 02:57:39 +0100

The company's OA system needs to query the data from the Activiti table when querying the function of the process I initiated


When the interface becomes slower and slower, we will respond to this problem

So I'm going to rewrite the company's query process function and give it a major system upgrade: you can search keywords in full text. So I chose to use ElasticSearch, a search engine. I have written several articles about ES before. You can go and have a look:

On the first day of the new year, the boss asked me to upgrade the ElasticSearch version. I said I had to pay more

My colleagues said that Mysql is used for keyword query. I go up like a pressure cooker. Isn't it fragrant to use ElasticSearch?

ElasticSearch benchmarking Mysql, who can take the lead?

In fact, there are many methods to call es, among which es provides java related APIs to use, but I choose to use the original way of http to operate es.
It can be divided into several ways. You can use native http to encapsulate it into a tool class for use. Another way is that if your company uses spring cloud, you can use its component Feign to call es, which will be more elegant and advanced (in fact, there is no)

@FeignClient(name = "EIP-MAPP-SEARCH-SERVER", url = "${elsearch.url}", fallbackFactory = ElasticHystrix.class)
public interface ElasticSearchApi {

    @PostMapping("/{index}/_doc/{id}")
    Object insertData(@PathVariable(name = "index") String index, @PathVariable(name = "id") String id, @RequestBody JSONObject jsonObject);

    @GetMapping("/{index}/_search")
    Object indexSearch(@PathVariable(name = "index") String index, @RequestBody JSONObject jsonObject);

    @GetMapping("/{aliases}/_search")
    Object aliasesSearch(@PathVariable(name = "aliases") String aliases, @RequestBody JSONObject jsonObject);

    @PutMapping("/{index}")
    Object insertAliases(@PathVariable(name = "index") String index, @RequestBody JSONObject jsonObject);

    @RequestMapping(value = "/{index}",method = RequestMethod.HEAD)
    Object checkIndex(@PathVariable(name = "index") String index);

    @PostMapping("/{index}/_doc/{id}/_update")
    Object updateData(@PathVariable(name = "index") String index, @PathVariable(name = "id") String id, @RequestBody JSONObject jsonObject);


}

Then configure the address of es server in the configuration file of yml (this can be customized)

elsearch:
  process:
    enable: true
  url:
    http://10.123.236.106:9200

If you don't know how to install es, please refer to my installation tutorial: (nanny teaching)

Install ElasticSearch and Ik word splitter on Linux (detailed version of graphic explanation)

After that, we can insert data into the business. With the data, we can query naturally. However, since I use the method of JsonObject to transfer parameters, we need to encapsulate it into the original parameters to transfer parameters when querying, just like the following two examples

Packaging method

 private ResultUtils getResultUtils(PageRequestDTO<OAElasticSearchParamDto> pageRequestDTO, String type, String queryParam) {
        JSONObject jsonObject = getPage(pageRequestDTO);
        //Query parameters
        OAElasticSearchParamDto params = pageRequestDTO.getParams();
        String keyWord = params.getKeyWord();
        //Time of submission and completion of approval
        List<ColumnOption> optionList = pageRequestDTO.getOptionList();
        JSONArray timeList = new JSONArray();
        if (!optionList.isEmpty()) {
            timeList = getTimeList(optionList);
        }
        //Query by keyword
        JSONObject elasticMatch = null;
        if (!StringUtils.isEmpty(keyWord)) {
            elasticMatch = getJsonObject(keyWord, "elasticData", "match");
        }
        //My inquiry
        UserInfoDTO user = UserUtils.getActiveUser().getUser();
        String userId = user.getPsnCode();
        if ("handledUser".equals(type)) {
            userId = "*" + userId + "*";
        }
        JSONObject myStartMatch = getJsonObject(userId, type, queryParam);
        //Task status
        return getResultUtils(pageRequestDTO, jsonObject, params, elasticMatch, myStartMatch, timeList);
    }

Packaging method

private ResultUtils getResultUtils(PageRequestDTO<OAElasticSearchParamDto> pageRequestDTO, JSONObject jsonObject,
                                       OAElasticSearchParamDto params, JSONObject elasticMatch, JSONObject myHandleJson, JSONArray timeList) {
        String taskStatus = pageRequestDTO.getParams().getTaskStatus();
        //Combined query
        JSONArray mustJsonArray = new JSONArray();
        mustJsonArray.add(elasticMatch);
        mustJsonArray.add(myHandleJson);
        mustJsonArray.addAll(timeList);
        if (!StringUtils.isEmpty(taskStatus)) {
            JSONObject taskStatusMatch = getJsonObject(taskStatus, "taskStatus", "match");
            mustJsonArray.add(taskStatusMatch);
        }
        JSONObject mustJson = new JSONObject();
        mustJson.put("must", mustJsonArray);
        // "bool" encapsulation
        JSONObject boolJson = new JSONObject();
        boolJson.put("bool", mustJson);

        jsonObject.put("query", boolJson);
        //Exclude return fields
        JSONObject excludes = new JSONObject();
        excludes.put("excludes", "elasticData");
        jsonObject.put("_source", excludes);
        //Microservice name
        String serveName = params.getServeName();
        Object search = elasticSearchApi.aliasesSearch(serveName, jsonObject);
        return ResultUtils.success(search);
    }

Call method

@Override
    public ResultUtils myStart(PageRequestDTO<OAElasticSearchParamDto> pageRequestDTO) {
        return getResultUtils(pageRequestDTO, "taskStartUser", "match");
    }


    /**
     * My to-do list
     *
     * @Param: [pageRequestDTO]
     * @return: com.lydsoft.eip.mapp.common.utils.ResultUtils
     * @Author: MaSiyi
     * @Date: 2022/1/26
     */
    @Override
    public ResultUtils myHandling(PageRequestDTO<OAElasticSearchParamDto> pageRequestDTO) {
        return getResultUtils(pageRequestDTO, "assignee", "match");
    }


    /**
     * I handled it
     *
     * @Param: [pageRequestDTO]
     * @return: com.lydsoft.eip.mapp.common.utils.ResultUtils
     * @Author: MaSiyi
     * @Date: 2022/1/26
     */
    @Override
    public ResultUtils myHandled(PageRequestDTO<OAElasticSearchParamDto> pageRequestDTO) {
        return getResultUtils(pageRequestDTO, "handledUser", "wildcard");
    }

Here's another tip: idea can extract repeated code fragments to form a new method, so we can steal laziness when writing code and hand over the extraction method to idea.

The specific shortcut key is Ctrl+Atl+M

After using es, these interfaces will query data directly from es. We only need to save the data in es at the same time when creating a new process. In this way, the speed will be much faster!!

Today is the summary of this article. If you learn some fur from it, I hope to encourage the blogger and give the blogger a praise collection and attention, which is the biggest driving force for the blogger to continue to create!!!

Topics: Big Data ElasticSearch search engine