Installation and configuration of elasticsearch and kibana in docker environment

Posted by brandonr on Wed, 09 Feb 2022 04:13:36 +0100

Introduction to Elasticsearch

Elasticsearch is a distributed, high expansion, high real-time search and data analysis engine. It can easily make a large number of data have the ability of search, analysis and exploration

  • Applicable scenarios of Elasticsearch
  1. abroad
  • Wikipedia, similar to Baidu Encyclopedia, full-text retrieval, highlighting, search recommendation
  • The Guardian (foreign news website), user behavior log (click, browse, collect, comment) + social network data, data analysis
  • Stack Overflow (foreign program exception discussion forum)
  • GitHub (open source code management)
  • E-commerce website, search goods
  • Log data analysis, logstash collection log, ES complex data analysis (ELK technology, elasticsearch+logstash+kibana)
  • Commodity price monitoring website
  • BI system, Business Intelligence, Business Intelligence.
  1. domestic
  1. On site search (e-commerce, recruitment, portal, etc.)
  2. IT system search (OA, CRM, ERP, etc.)
  3. Data analysis (a popular usage scenario of ES)
  • Characteristics of Elasticsearch
  1. It can be used as a large-scale distributed cluster (hundreds of servers) technology to process PB level data and serve large companies; It can also run on a single machine to serve small companies
  2. Elasticsearch is not a new technology. It mainly combines full-text retrieval, data analysis and distributed technology to form a unique ES; lucene (full text search), commercial data analysis software (also available), distributed database (mycat)
  3. For users, it is used out of the box. It is very simple. As a small and medium-sized application, the ES can be deployed directly in 3 minutes and can be used as a system in the production environment. The amount of data is small and the operation is not too complex
  4. The function of database is not enough in many fields (transaction and various online transaction operations); Special functions, such as full-text retrieval, synonym processing, relevance ranking, complex data analysis, near real-time processing of massive data; Elastic search, as a supplement to the traditional database, provides many functions that the database cannot provide
  • Functions of Elasticsearch
  • Distributed search engine and data analysis engine
  • Full text retrieval, structured retrieval, data analysis
  • Processing massive data in near real time

reference: Elasticsearch functions, applicable scenarios and features

Installation configuration of Elasticsearch

  1. docker gets the image of es
  docker pull elasticsearch:7.12.1
  1. Create es related file directories on the host
   mkdir /docker/es 
   mkdir /docker/es/conf  #configuration file
   mkdir /docker/es/data  #data
   mkdir /docker/es/plugins #Extended directory
  1. create profile
touch /docker/es/conf/elasticsearch.yml
vim /docker/es/conf/elasticsearch.yml
############### Write the following configuration  ##############
#Cluster name my-application 
#Node name es-node-1 
#Storage directory for data and logs /usr/share/elasticsearch/data 
path.logs: /usr/share/elasticsearch/logs
##Set the bound ip to, and then any computer node can access it 
http.port: 9200 
##Set the names of all nodes in the cluster. This node name is modified before. Of course, you can also use the default one. At present, it is a stand-alone node, and you can put it into one node 
cluster.initial_master_nodes: ["es-node-1"]
  1. Build container
docker run -p 9200:9200 -d --name es -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -v /docker/es/conf/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /docker/es/data:/usr/share/elasticsearch/data -v /docker/es/plugins:/usr/share/elasticsearch/plugins --privileged=true elasticsearch:7.12.1
  1. Build exception handling

dockers logs es or docker logs -f es view the build container logs
Exception: Max virtual memory areas VM max_ map_ count [65530] is too low,increase to at least [262144]

  • solve:

Modify the configuration sysctl conf vi /etc/sysctl. conf
Add the following at the end of the line VM max_ map_ count=655300
Execute the command sysctl -p
Restart es container

  • The browser accesses the installed elasticsearch. The access address is

kibana installation configuration

  1. Pull kibana image
docker pull kibana:7.12.1
  1. kibana's profile
mkidr /docker/kibana #Create Host Directory
mkdir /docker/kibana/conf 
touch /docker/kibana/conf/kibana.yml #create profile 
  1. kibana.yml file content kibana "" 
 elasticsearch.hosts: ["http://es address: 9200 "] 
 xpack.monitoring.ui.container.elasticsearch.enabled: true
  1. Building containers for kibana
docker run -p 5601:5601 -d --name kibana -v /docker/kibana/conf/kibana.yml:/usr/share/kibana/config/kibana.yml --privileged=true kibana:7.12.1
  • kibana access address

Topics: Docker ElasticSearch kibana