Elasticsearch client and API usage in SpringBoot

Posted by nashirak on Wed, 09 Mar 2022 10:51:25 +0100

Official documents: Java Transport Client (deprecated) [7.17] | Elastic

1. Create client

(1)TransportClient

The TransportClient client will be deprecated in ES version 7.0 and completely removed in version 8.0

Create steps:

//Directly in http://127.0.0.1:9200/ You can see the cluster corresponding to es in_ name
Settings settings= Settings.builder()
    .put("cluster.name","Here is es Corresponding cluster_name").build(); 
TransportClient transportClient=new PreBuiltTransportClient(settings);
transportClient.addTransportAddress(new 
          TransportAddress(InetAddress.getByName("127.0.0.1"),9300));

Then you can operate es through the transportClient.

(2)RestHighLevelClient

There are also corresponding RestLowLevelClient low-level clients, but there are not many users.

RestHighLevelClient is officially launched. It uses Http to connect the query results, so it needs port 9200. It's RestFul.

RestHighLevelClient restHighLevelClient(){
    RestHighLevelClient client = new RestHighLevelClient(
        RestClient.builder(
            new HttpHost("127.0.0.1", 9200, "http")));
    return client;
}

Then you can operate es through the client.

These are the two longest clients.

(3)Jest

Github: GitHub - searchbox-io/Jest: Elasticsearch Java Rest Client.

If different versions of ES clusters are used in the project, using native clients will be a problem. If the version of ES is not a problem, it is better to use the native client. Es does not have a Rest client, so the reason for Jest.

Maven Repository: Index of /groups/public/io/searchbox/jest From 5 JDK8 is the minimum requirement from version X

/**
     * Simply build a Jestclient
     *
     * @param serverList es List of cluster machines. The shape of each machine is as follows: http://xxx.xxx.xxx:9200
     * @return
     */
public static JestHttpClient getJestHttpClient(List<String> serverList) {
    JestClientFactory jestClientFactory = new JestClientFactory();

    HttpClientConfig.Builder httpClientConfigBuilder = new                                          HttpClientConfig.Builder(serverList);

    jestClientFactory.setHttpClientConfig(httpClientConfigBuilder.build());

    JestHttpClient jestHttpClient = (JestHttpClient) jestClientFactory.getObject();

    return jestHttpClient;
}

HttpClientConfig

parameter	explain	remarks
serverList	ES cluster machine list
isMultiThreaded	Whether to use HttpClient connection pool
isDiscoveryEnabled	Enable node detection	It is recommended to use this parameter to detect and update the machine list when the machine in the cluster goes offline
discoveryFilter
isRequestCompressionEnabled	Enable GZIP compression request	Requires ES node to set http compression
connTimeout	connTimeout of HttpClient
readTimeout	readTimeout of HttpClient
discoveryFrequency	Frequency of node detection
maxConnectionIdleTime
discoveryFrequencyTimeUnit	Frequency unit of node detection	TimeUnit
maxConnectionIdleTimeDurationTimeUnit
gson
defaultSchemeForDiscoveredNodes
maxTotalConnection	The maximum number of connections in the HttpClient connection pool
defaultMaxTotalConnectionPerRoute	The default maximum number of connections per route in the HttpClient connection pool
maxTotalConnectionPerRoute	Maximum number of connections routed by HttpClient connection pool	The default value of defaultMaxTotalConnectionPerRoute will be overridden
credentialsProvider
sslSocketFactory
plainSocketFactory
httpRoutePlanner
proxyAuthenticationStrategy
httpIOSessionStrategy
httpsIOSessionStrategy
preemptiveAuthTargetHosts

2.SearchRequest,BulkRequest..

Both are built on the premise of RestHighLevelClient advanced restful style client. Get the client above

@Autowired
    private RestHighLevelClient restHighLevelClient;

    //PUT testCreate
    @Test
    void testCreate() throws IOException {
        //1. Create index request
        CreateIndexRequest createIndexRequest = new CreateIndexRequest("test_create");
        //2. Execute the create request {IndicesClient, and get the response after the request
        CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(createIndexRequest, RequestOptions.DEFAULT);
        System.out.println(createIndexResponse);
    }

    //Test to obtain the index and judge whether it exists
    @Test
    void getEs() throws IOException {
        GetIndexRequest test_create = new GetIndexRequest("test_create");
        boolean exists = restHighLevelClient.indices().exists(test_create, RequestOptions.DEFAULT);
        System.out.println(exists);
    }

    //Test delete index
    @Test
    void deleteEs() throws IOException {
        DeleteIndexRequest test_create = new DeleteIndexRequest("test_create");
        AcknowledgedResponse delete = restHighLevelClient.indices().delete(test_create, RequestOptions.DEFAULT);
        System.out.println(delete);
    }

    //Test add document
    @Test
    void AddEsWd() throws IOException {
        //Create entity class object
        User user = new User("I'm test integration java",12);
        //Create request
        IndexRequest test_create = new IndexRequest("test_create");
        //Rule creation
        test_create.id("1");
        test_create.timeout(TimeValue.timeValueSeconds(1));
        //If it exceeds 1s, it will not be executed
        test_create.timeout("1s");
        //Fast JSON is used here to put the data into the request, but hutool tool class is recommended
        IndexRequest source = test_create.source(JSON.toJSONString(user), XContentType.JSON);
        //client poke request
        IndexResponse index = restHighLevelClient.index(source, RequestOptions.DEFAULT);
        //Get the corresponding results
        System.out.println(index.toString());   //IndexResponse[index=test_create,type=_doc,id=1,version=1,result=created,seqNo=0,primaryTerm=1,shards={"total":2,"successful":1,"failed":0}]
        System.out.println(index.status()); //CREATED
    }

    //Test acquisition document
    @Test
    void getWd() throws IOException {
        GetRequest test_create = new GetRequest("test1", "1");
        //Do not get returned_ source context is more efficient
        test_create.fetchSourceContext(new FetchSourceContext(false));
        test_create.storedFields("_none_");

        boolean exists = restHighLevelClient.exists(test_create, RequestOptions.DEFAULT);
        System.out.println(exists);
    }

    //Test to get the information of the document
    @Test
    void getWdXx() throws IOException {
        GetRequest test_create = new GetRequest("test_create", "1");
        GetResponse exists = restHighLevelClient.get(test_create, RequestOptions.DEFAULT);
        System.out.println(exists);//Return all contents
        System.out.println(exists.getSource());//Get the information in source {name = I'm testing integration java, age=12}
        System.out.println(exists.getVersion());//Get version
        System.out.println(exists.getSourceAsString());//Print the content of the document {"age":12,"name": "I'm testing integration java"}
    }

    //Update the information of the document
    @Test
    void updateWdXx() throws IOException {
        UpdateRequest test_create = new UpdateRequest("test_create","1");
        User user = new User("Update test", 12);
        test_create.doc(JSON.toJSONString(user),XContentType.JSON);
        UpdateResponse update = restHighLevelClient.update(test_create, RequestOptions.DEFAULT);
        System.out.println(update);
    }

    //Delete document record
    @Test
    void deleteWdXx() throws IOException {
        DeleteRequest test_create = new DeleteRequest("test_create", "1");
        DeleteResponse delete = restHighLevelClient.delete(test_create, RequestOptions.DEFAULT);
        System.out.println(delete);
    }

    //Real projects need to insert data in batches
    @Test
    void testBulkRequest() throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
        bulkRequest.timeout("10s");
        ArrayList<User> users = new ArrayList<>();
        users.add(new User("test1",1));
        users.add(new User("test2",2));
        users.add(new User("test3",3));
        users.add(new User("test4",4));
        users.add(new User("test5",5));
        //Batch request
        int i = 1;
        for (User user : users){
            i++;
            bulkRequest.add(new IndexRequest("test_create")
                    .id(""+i) //If you don't write an id, a random id will be generated
                    .source(JSON.toJSONString(user),XContentType.JSON));
        }
        BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
        System.out.println(bulk);
        System.out.println(bulk.hasFailures());//Fail false success
    }

    //query
    @Test
    void testSearch() throws IOException {
        SearchRequest searchRequest = new SearchRequest("test_create");
        //Build search criteria
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        //Create a query constructor for grouping queries
        MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name","test");
        searchSourceBuilder.query(matchQueryBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        searchRequest.source(searchSourceBuilder);
        //RequestOptions.DEFAULT default
        SearchResponse search = restHighLevelClient.search(searchRequest,RequestOptions.DEFAULT);
        System.out.println(search.getHits());
        System.out.println("=======================");
        for (SearchHit hit : search.getHits()){
            System.out.println(hit.getSourceAsString());
        }
    }

3.ElasticsearchRepository<T, String>

The TransportClient client can be used. RestHighLevelClient should be ok if it has not been tested.

The es operation references the library.

T represents the entity class corresponding to the index, which stores the fields to be returned in the index, the corresponding getset method and the construction method with and without parameters.

You can directly use the annotations in lombok to add the corresponding getset and construction methods

import lombok.*;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;

@Getter
@Setter
@ToString
@AllArgsConstructor     //Have reference
@NoArgsConstructor      //No reference
//@Document(indexName = "test", type = "_doc")
@Document(indexName = "Index name", type = "type")
public class test implements Serializable {
    @Id
    private String id;// Primary key id in index

    private String name; 

    private Date birthday;
}

Then create a dao layer to implement elasticsearchrepository < test, string >

To rewrite the method you need, you can use service to invoke the dao layer to search for it.

import com.hyq.test;
import org.elasticsearch.index.query.QueryBuilder;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.core.query.SearchQuery;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

import java.util.Optional;

public interface test extends ElasticsearchRepository<ELKZFJD, String> {
    @Override
    public Iterable<test> findAll();

    @Override
    Optional<test> findById(String s);

    @Override
    void deleteById(String s);

    @Override
    void deleteAll(Iterable<? extends test> entities);

    @Override
    void deleteAll();

    @Override
    Page<test> search(SearchQuery searchQuery);

    @Override
    Page<test> search(QueryBuilder queryBuilder, Pageable pageable);

}

Class, such as page < test > Search (querybuilder, querybuilder, pageable pageable);

It can be called in the business layer. Then use * * querybuilder (boolquerybuilder, matchquerybuilder..) Write query logic. And set Pageable for paging and sorting. Pass in these two parameters to get the query result.

4.QueryBuilder

Common methods:

//Perform a single value exact match termQuery("key", obj) exact match
QueryBuilders.termQuery("name","I am Chinese,")
    
//Perform fuzzy query, multiMatchQuery("key.keyword", "*field1 *"); Wildcards can be used Whether keyword is added? Check the FAQ of Es above
QueryBuilders.wildcardQuery("name"/"name.keyword","*I'm Chinese*")
    
//Perform multiple value exact matching termsQuery("key", obj1, obj2...) Match multiple values at once
String[] a = {"a","ab","ac","qq"}
QueryBuilders.termsQuery("name",a)
    
//Wildcards cannot be used when fuzzy query is compared with wildcardQuery
QueryBuilders.matchQuery("name","I am")
    
//matchPhraseQuery for Chinese exact matching
queryBuilders.matchPhraseQuery("key", value)
    
//Make Boolean query
QueryBuilders.boolQuery()
    
//Fuzzy query
QueryBuilders.boolQuery()
    
//Query the range. gte is greater than or equal to, lte is less than or equal to, gt is greater than or less than
QueryBuilders.rangeQuery("age")
    .gte(jsonObjectEq.get("1"))
    .lte(jsonObjectEq.get("10")))
......

(1)BoolQueryBuilder

Multi condition judgment query can be performed

BoolQueryBuilder builder = QueryBuilders.boolQuery();
//Must where the conditions must be met and
builder.must(queryBuilders.matchPhraseQuery("key", value));
//mustNot where the condition must not be met
builder.mustNot(queryBuilders.matchPhraseQuery("key", value));
//should one of the conditions is satisfied or
builder.should(queryBuilders.matchPhraseQuery("key", value));
builder.should(QueryBuilders.matchQuery("name","I am"));

When must and should are used at the same time, minumum_should_match defaults to 0. So should won't work.

boolQueryBuilder.minimumShouldMatch(1) can solve this problem by changing it to 1.

Then put it into NativeSearchQueryBuilder withQuery(builder). Use native searchquerybuilder for more complex queries.

(2)NativeSearchQueryBuilder

NativeSearchQuery is a native query condition class, which is used to match with some native query methods of ES to realize some complex queries.

Native searchquerybuilder can perform complex query settings.

 NativeSearchQueryBuilder nativeSearchQueryBuilder = new NativeSearchQueryBuilder();
        //Set search criteria to build
        nativeSearchQueryBuilder.withQuery(builder);
        //Set paging to build
        nativeSearchQueryBuilder.withPageable(page);
        //Set the sorting to the construction (I'm not good at it, so I'd better set it directly in Pageable and put it in the upper page)
        nativeSearchQueryBuilder.withSort(sort);
        //Production NativeSearchQuery
        NativeSearchQuery query = nativeSearchQueryBuilder.build();

Then put the query into the search method or searchrequestbuilder Query(). Perform a copy query and obtain the corresponding return results.

5.Pageable

It mainly uses its implementation class PageRequest.

import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable;

//Page refers to the current number of pages, length refers to the number of data on one page, and sort refers to sorting (see the next section for details)
Pageable page1 = PageRequest.of(page, length, sort);
//You can also not set sorting
Pageable page1 = PageRequest.of(page, length);

6.Sort

import org.springframework.data.domain.Sort;

Sort sort = Sort.by(Sort.Direction.ASC,"Fields to sort");
Sort sort = Sort.by(Sort.Direction.DESC,"update_time")

Topics: Java ElasticSearch Spring Boot

Programmer Think