1, elasticsearch deployment

1. Pull image

docker pull elasticsearch:7.12.1

2. Create host file path

mkdir -p /data/elasticsearch/config
mkdir -p /data/elasticsearch/data
mkdir -p /data/elasticsearch/logs
mkdir -p /data/elasticsearch/plugins
# Give permission (sometimes an error will be reported when the permission is insufficient)
chmod 777 -R /data/elasticsearch

3. Edit profile

vi /data/elasticsearch/config/elasticsearch.yml

Profile content

cluster.name: "docker-cluster"
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"

4. Run docker instance

docker run -d --name elasticsearch --restart=always -p 9200:9200 -p 9300:9300  -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms64m -Xmx128m" -v /data/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /data/elasticsearch/data:/usr/share/elasticsearch/data -v /data/elasticsearch/plugins:/usr/share/elasticsearch/plugins -v /data/elasticsearch/logs:/usr/share/elasticsearch/logs elasticsearch:7.12.1

Parameter analysis

discovery. Type = single node is a single instance operation

5. Verify whether es is normal

Check whether the instance is normal

docker ps  -a

Access

http://122.152.218.235:9200/

Correct display

{
  "name" : "5fa8ae6a746d",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "6381LYBCQF2RwTIo-hi8jA",
  "version" : {
    "number" : "7.12.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "3186837139b9c6b6d23c3200870651f10d3343b7",
    "build_date" : "2021-04-20T20:56:39.040728659Z",
    "build_snapshot" : false,
    "lucene_version" : "8.8.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

5. es visualization tool

Pull image

docker pull mobz/elasticsearch-head:5

Running instance

docker run -d --name=elasticsearch-head --restart=always -p 9100:9100 mobz/elasticsearch-head:5

Check whether the installation is normal

Check whether the instance is normal

docker ps  -a

Access

http://122.152.218.235:9100/

Usually, there is no data at this time, and you can only see the default home page

Alternative scheme (the test effect is the same at present)

docker pull mobz/elasticsearch-head:5-alpine

docker run -d  --name=elasticsearch-head  --restart=always  -p 9100:9100  docker.io/mobz/elasticsearch-head:5-alpine

2, Install logstash

1. Pull image

docker pull logstash:7.12.1

2. Start logstash instance

docker run -d --name=logstash logstash:7.12.1

Wait 30 seconds to view the log

docker logs -f logstash

If the following information appears, the startup is successful.

[2021-05-18T15:12:01,224][INFO ][org.logstash.beats.Server] Starting server on port: 5044

3. Copy data and grant permission

Copy the logstash file from the instance

docker cp logstash:/usr/share/logstash /data/

Create profile path

mkdir /data/logstash/config/conf.d

Assign permissions (sometimes running with insufficient permissions will report an error)

chmod 777 -R /data/logstash

4. Modify logstash configuration file

Modify the elasticsearch address in the configuration file

vi /data/logstash/config/logstash.yml

The complete contents are as follows:

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: ["http://172.17.0.6:9200"]
path.config: /usr/share/logstash/config/conf.d/*.conf
path.logs: /usr/share/logstash/logs

Modify the elasticsearch address according to the actual situation

4. Add the configuration file of logstash collection log

Create a new file syslog Conf to collect / var/log/messages

vi /data/logstash/config/conf.d/syslog.conf

The complete contents are as follows:

input {
  file {
    #label
    type => "systemlog-localhost"
    #Collection point
    path => "/var/log/messages"
    #Start collection point
    start_position => "beginning"
    #The scanning interval is 1s by default and 5s is recommended
    stat_interval => "5"
  }
}

output {
  elasticsearch {
    hosts => ["172.17.0.6:9200"]
    index => "logstash-system-localhost-%{+YYYY.MM.dd}"
 }
}

Modify the elasticsearch address according to the actual situation

5. Set log file read permissions

chmod 644 /var/log/messages
chmod 777 -R /data/logstash

6. Restart the logstash instance

Delete the instance created above that does not have a mapping host file

docker rm -f logstash

Run the new logstash instance

docker run -d  --name=logstash  --restart=always  -p 5044:5044   -v /data/logstash:/usr/share/logstash  -v /var/log/messages:/var/log/messages  logstash:7.12.1

7. After the restart is completed, access elasticsearch head

http://122.152.218.235:9100/

3, kibana deployment

1. Pull image

docker pull kibana:7.12.1

2. Create profile

vi /data/kibana/config/kibana.yml

Complete configuration content:

#
# ** THIS IS AN AUTO-GENERATED FILE **
#

# Default Kibana configuration for docker target
server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://172.17.0.6:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true
i18n.locale: "zh-CN"

3. Running instance

docker run -d  --name=kibana --restart=always  -p 5601:5601  -v /data/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml kibana:7.12.1

4. Check whether the installation is normal

Check whether the instance is normal

docker ps  -a

Access connection

http://122.152.218.235:5601/

Usually, there is no data at this time, and you can only see the default home page

5. Display log data by creating index mode

The data is the log file obtained by logstash

• Click [Discover] on the left menu
• After entering the page, click [index mode] under the [kibana] menu on the left
• After entering the page, click [create index mode], enter [logstash system localhost - *] and click next
• After entering the page, click [event field], select [@ timestamp], and click [create index mode]
• Click the [Discover] menu again to view the effect

Reference website

docker installation kibana

https://blog.csdn.net/shykevin/article/details/108272260

Elastic China community official blog

https://elasticstack.blog.csdn.net/

ELK log analysis system, overview and deployment

https://blog.csdn.net/Jun____________/article/details/116693522

Supplement docker knowledge

When using the docker container, sometimes vim is not installed inside, and you will be prompted when you click the vim command

vim: command not found

You need to install vim yourself

# Synchronize / etc / apt / sources List and / etc / apt / sources list. The index of the source listed in D
apt-get update

# Install vim
apt-get install -y vim