docker 安装 elk 环境

发表于 | 更新于 | 分类于 | 阅读次数：
| 字数： 3.2k | 时长 ≈ 3 分钟
标签

安装

从 Docker 仓库中拉取镜像

1
docker pull sebp/elk

拉取指定版本镜像

1
docker pull sebp/elk:671

启动容器

使用docker-compose

docker-compose.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
elk:
  image: sebp/elk:671
  ports:
    # Elasticsearch
    - "9200:9200"
    # Logstash
    - "5044:5044"
  	# Kibana
    - "5601:5601"
  volumes:
    # elasticsearch
    # 持久日志数据
    - "/data/docker/elk/elasticsearch:/var/lib/elasticsearch"
    # logstash
    ## config/logstash.yml, config/jvm.options, config/pipelines.yml
    ## logstash-plugin：/opt/logstash/bin
    - "/data/docker/elk/logstash/:/opt/logstash"
    ## 01-lumberjack-input.conf, 02-beats-input.conf
    - "/data/docker/elk/logstash/conf.d:/etc/logstash/conf.d"
    # kibana
    ## kibana-plugin：/opt/kibana/bin
    - "/data/docker/elk/kibana/:/opt/kibana"
  restart: always
  environment:
    - "bootstrap.memory_lock=true"
    - "ES_JAVA_OPTS=-Xss256k"
1
docker-compose up -d

启动异常

  1. m.max_map_count [65530] is too low

    1
    2
    3
    4
    2020-08-21T04:04:58,114[o.e.b.BootstrapChecks    ] [8pUsAbG] bound or publishing to a non-loopback address, enforcing bootstrap checks
    2020-08-21T04:04:58,120[o.e.b.Bootstrap          ] [8pUsAbG] node validation exception
    [1] bootstrap checks failed
    [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

配置 /etc/sysctl.conf

1
2
$ vi /etc/sysctl.conf
vm.max_map_count=262144

生效

1
$ sysctl -p

验证

1
$ sysctl -a|grep vm.max_map_count

进入容器命令行

1
2
3
4
# centos
docker exec -it <container-name> /bin/bash
# alpine
docker exec -it <container-name> sh

Elasticsearch 索引定时清理

  • elasticsearch-curator 安装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 安装 curator 源
$ rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

# 编辑 curator yum 源配置
$ vim /etc/yum.repos.d/curator.repo

[curator-5]
name=CentOS/RHEL 7 repository for Elasticsearch Curator 5.x packages
baseurl=https://packages.elastic.co/curator/5/centos/7
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

# 安装 curator
$ yum install elasticsearch-curator -y
  • 配置 config.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ mkdir -p /data/ELKStack/curator
$ vim /data/ELKStack/curator/config.yml

lient:
  hosts:
    - 172.16.1.3
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 150
  master_only: False

logging:
  loglevel: INFO
  logfile:
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']
  • 配置 action.yml 清理规则

/data/ELKStack/curator/action.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 60 days. Ignore the error if the    filter does not result in an actionable list of indices    (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      # 保留 kibana|json|monitoring|metadata 不被清理
      value: '^((?!(kibana|json|monitoring|metadata)).)*$'
    - filtertype: age
      source: creation_date
      direction: older
      #timestring: '%Yi-%m-%d'
      unit: days
      unit_count: 60
  • 设置计划任务
1
$ crontab -e  0 0 * * * /usr/bin/curator --config /data/ELKStack/curator/config.yml /data/ELKStack/curator/action.yml 1>> /tmp/curator.log 2>&1
  • 本文作者： forever杨
  • 本文链接： https://blog.yl-online.top/posts/6a0c4822.html
  • 版权声明： 本博客所有文章除特别声明外，均采用 BY-NC-SA 许可协议。如果文章内容对你有用，请记录到你的笔记中。本博客站点随时会停止服务，请不要收藏、转载！