docker日志采集elk efk fluentd采集

本文主要是介绍docker日志采集elk efk fluentd采集，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

第一步: Create docker-compose.yml

创建docker-合成.yml对于Docker Compose。Docker Compose是一个用于定义和运行多容器Docker应用程序的工具。通过下面的YAML文件，您可以通过一个命令创建和启动所有服务（在本例中是Apache、Fluentd、Elasticsearch、Kibana）：

version: '3'
services:fluentd:build: ./fluentdvolumes:- ./fluentd/conf:/fluentd/etclinks:- "elasticsearch"ports:- "24224:24224"- "24224:24224/udp"elasticsearch:image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0environment:- "discovery.type=single-node"expose:- "9200"ports:- "9200:9200"kibana:image: kibana:7.2.0links:- "elasticsearch"ports:- "5601:5601"

如果你已经安装es及kibana,则可使用下面的配置

version: '3'
services:fluentd:build: .volumes:- ./fluentd/conf:/fluentd/etcprivileged: trueports:- "24224:24224"- "24224:24224/udp"environment:- TZ=Asia/Shanghairestart: alwayslogging:driver: "json-file"options:max-size: 100mmax-file: "5"

使用Config+插件创建Fluentd映像

使用fluentd官方Docker映像创建包含以下内容的fluentd/Dockerfile；然后安装Elasticsearch插件：

# fluentd/DockerfileFROM fluent/fluentd:v1.6-debian-1
USER root
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "4.2.0"]
USER fluent

然后，创建Fluentd配置文件fluentd/conf/fluent.conf。转发输入插件从Docker日志驱动程序接收日志，elasticsearch输出插件将这些日志转发到elasticsearch。

# fluentd/conf/fluent.conf<source>@type forwardport 24224bind 0.0.0.0
</source>
<match docker.**>@type copy<store>@type elasticsearchhost elasticsearchport 9200logstash_format truelogstash_prefix sharechef-logslogstash_dateformat %Y%m%dinclude_tag_key truetype_name access_logflush_interval 1sinclude_tag_key truereconnect_on_error truereload_on_failure truereload_connections falsetag_key @log</store>
</match>

第二步：启动容器

$ docker-compose up

第三步: 创建验证nginx

docker run -dit -p 80:80\--log-driver=fluentd \--log-opt fluentd-address=localhost:24224 \--log-opt tag="docker.``.`Name`" \--name nginx \nginx

curl 127.0.0.1#查看日志
docker logs fluentd容器id127.0.0.1:5601
登录kibana
GET _cat/indices#查看索引
GET /docker-logs-20200925/_search
{"query": {"match_all": {}}
}