filebeat作为daemonSet

2024-03-29 23:52

文章标签 filebeat daemonset

本文主要是介绍filebeat作为daemonSet，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

创建 Filebeat 服务账户和 ClusterRole

apiVersion: v1
kind: Namespace
metadata:name: logging
---
apiVersion: v1
kind: ServiceAccount
metadata:name: filebeatnamespace: logginglabels:k8s-app: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: filebeatnamespace: logginglabels:k8s-app: filebeat
rules:
- apiGroups: [""]resources:- namespaces- pods- nodesverbs:- get- watch- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: filebeatnamespace: logging
subjects:
- kind: ServiceAccountname: filebeatnamespace: logging
roleRef:kind: ClusterRolename: filebeatapiGroup: rbac.authorization.k8s.io

2. 创建 Filebeat ConfigMap

kind: ConfigMap
apiVersion: v1
metadata:name: filebeat-confignamespace: logginglabels:k8s-app: filebeat
data:filebeat.yml: |-filebeat.inputs:- type: filestreamenabled: truepaths:- /home/leve/logs/*/*.log- /var/log/leve/test/*/*.logtags: ["test-log"]fields:source: test-logmultiline.pattern: '^\d{4}-\d{2}-\d{2}'multiline.negate: truemultiline.match: afterindex: filebeat-service-test- type: logenabled: truepaths:- /var/log/leve/*/*.log- /var/log/leve/dev/*/*.logtags: ["dev-log"]fields:source: dev-logmultiline.pattern: '^\d{4}-\d{2}-\d{2}'multiline.negate: truemultiline.match: afterindex: filebeat-service-devprocessors:- add_host_metadata:- script:lang: javascriptsource: >function process(event) {try {var message = event.Get("message")var arr = message.match(/^(\d{4}-\d{2}-\d{2}T\S+)\s+([a-zA-Z]+)\s+\[([a-zA-Z]+)(.*)/)var date = arr[1]var level = arr[2]var service = arr[3]var content = arr[4]var fields = event.Get("fields")fields["date"] = datefields["level"] = levelfields["service"] = servicefields["content"] = contentevent.Put("fields", fields)} catch(err) {var fields = event.Get("fields")fields["js_err"] = err.messageevent.Put("fields", fields)}}filebeat.config.modules:path: ${path.config}/modules.d/*.ymlreload.enabled: falseoutput.elasticsearch:hosts: ["es1.leve.com:9202", "es2.leve.com:9202", "es3.leve.com:9202"]username: "elastic"password: "123456"

3. 创建filebeat DaemonSet

apiVersion: apps/v1
kind: DaemonSet
metadata:name: filebeatnamespace: logginglabels:k8s-app: filebeat
spec:selector:matchLabels:k8s-app: filebeattemplate:metadata:labels:k8s-app: filebeatspec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: zoneoperator: ExistsserviceAccountName: filebeatterminationGracePeriodSeconds: 30hostNetwork: truednsPolicy: ClusterFirstWithHostNettolerations:- effect: NoSchedulekey: node-role.kubernetes.io/mastercontainers:- name: filebeatimage: elastic/filebeat:8.12.2args: ["-c", "/root/filebeat.yml","-e",]env:- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeNamesecurityContext:runAsUser: 0resources:limits:memory: 300Mirequests:cpu: 100mmemory: 100MivolumeMounts:- name: configmountPath: /root/filebeat.ymlreadOnly: truesubPath: filebeat.yml- name: datamountPath: /usr/share/filebeat/data- name: dev-logmountPath: /var/log/leve- name: test-logmountPath: /home/leve/logs- name: varlibdockercontainersmountPath: /var/lib/docker/containersreadOnly: truevolumes:- name: configconfigMap:defaultMode: 0600name: filebeat-config- name: varlibdockercontainershostPath:path: /var/lib/docker/containers- name: dev-loghostPath:path: /var/log/leve- name: test-loghostPath:path: /home/leve/logs- name: datahostPath:# When filebeat runs as non-root user, this directory needs to be writable by group (g+w).path: /var/lib/filebeat-datatype: DirectoryOrCreate
---