Introduction to Elasticsearch

Elasticsearch is a distributed, high expansion, high real-time search and data analysis engine. It can easily make a large number of data have the ability of search, analysis and exploration

• Applicable scenarios of Elasticsearch

1. abroad

• Wikipedia, similar to Baidu Encyclopedia, full-text retrieval, highlighting, search recommendation
• The Guardian (foreign news website), user behavior log (click, browse, collect, comment) + social network data, data analysis
• Stack Overflow (foreign program exception discussion forum)
• GitHub (open source code management)
• E-commerce website, search goods
• Log data analysis, logstash collection log, ES complex data analysis (ELK technology, elasticsearch+logstash+kibana)
• Commodity price monitoring website
• BI system, Business Intelligence, Business Intelligence.

1. domestic

1. On site search (e-commerce, recruitment, portal, etc.)
2. IT system search (OA, CRM, ERP, etc.)
3. Data analysis (a popular usage scenario of ES)

• Characteristics of Elasticsearch

1. It can be used as a large-scale distributed cluster (hundreds of servers) technology to process PB level data and serve large companies; It can also run on a single machine to serve small companies
2. Elasticsearch is not a new technology. It mainly combines full-text retrieval, data analysis and distributed technology to form a unique ES; lucene (full text search), commercial data analysis software (also available), distributed database (mycat)
3. For users, it is used out of the box. It is very simple. As a small and medium-sized application, the ES can be deployed directly in 3 minutes and can be used as a system in the production environment. The amount of data is small and the operation is not too complex
4. The function of database is not enough in many fields (transaction and various online transaction operations); Special functions, such as full-text retrieval, synonym processing, relevance ranking, complex data analysis, near real-time processing of massive data; Elastic search, as a supplement to the traditional database, provides many functions that the database cannot provide

• Functions of Elasticsearch

• Distributed search engine and data analysis engine
• Full text retrieval, structured retrieval, data analysis
• Processing massive data in near real time

reference: Elasticsearch functions, applicable scenarios and features

Installation configuration of Elasticsearch

1. docker gets the image of es

  docker pull elasticsearch:7.12.1

1. Create es related file directories on the host

   mkdir /docker/es 
   mkdir /docker/es/conf  #configuration file
   mkdir /docker/es/data  #data
   mkdir /docker/es/plugins #Extended directory

1. create profile

touch /docker/es/conf/elasticsearch.yml
vim /docker/es/conf/elasticsearch.yml
############### Write the following configuration  ##############
#Cluster name 
cluster.name: my-application 
#Node name
node.name: es-node-1 
#Storage directory for data and logs
path.data: /usr/share/elasticsearch/data 
path.logs: /usr/share/elasticsearch/logs
##Set the bound ip to 0.0.0.0, and then any computer node can access it 
network.host: 0.0.0.0 
#port
http.port: 9200 
##Set the names of all nodes in the cluster. This node name is modified before. Of course, you can also use the default one. At present, it is a stand-alone node, and you can put it into one node 
cluster.initial_master_nodes: ["es-node-1"]

1. Build container

docker run -p 9200:9200 -d --name es -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -v /docker/es/conf/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /docker/es/data:/usr/share/elasticsearch/data -v /docker/es/plugins:/usr/share/elasticsearch/plugins --privileged=true elasticsearch:7.12.1

1. Build exception handling

dockers logs es or docker logs -f es view the build container logs
Exception: Max virtual memory areas VM max_ map_ count [65530] is too low,increase to at least [262144]

• solve:

Modify the configuration sysctl conf vi /etc/sysctl. conf
Add the following at the end of the line VM max_ map_ count=655300
Execute the command sysctl -p
Restart es container

• The browser accesses the installed elasticsearch. The access address is 192.168.148.188:9200
  

kibana installation configuration

1. Pull kibana image

docker pull kibana:7.12.1

1. kibana's profile

mkidr /docker/kibana #Create Host Directory
mkdir /docker/kibana/conf 
touch /docker/kibana/conf/kibana.yml #create profile 

1. kibana.yml file content

 server.name: kibana 
 server.host: "0.0.0.0" 
 elasticsearch.hosts: ["http://es address: 9200 "] 
 xpack.monitoring.ui.container.elasticsearch.enabled: true

1. Building containers for kibana

docker run -p 5601:5601 -d --name kibana -v /docker/kibana/conf/kibana.yml:/usr/share/kibana/config/kibana.yml --privileged=true kibana:7.12.1

• kibana access address 192.168.148.188:5601