ECK是官方推出的在Kubernetes集群中的部署方式，其采用operator的方式进行部署。

部署operator

  1. kubectl apply -f https://download.elastic.co/downloads/eck/1.1.2/all-in-one.yaml

然后通过一下命令查看日志输出

  1. kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

可以通过一下命令查看部署情况

  1. # kubectl get pod -n elastic-system 
  2. NAME                      READY   STATUS    RESTARTS   AGE
  3. elastic-operator-0        1/1     Running   0          19m
  4. # kubectl get svc -n elastic-system 
  5. NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
  6. elastic-webhook-server    ClusterIP   10.102.3.82      <none>        443/TCP    19m

部署ElasticSearch

编写YAML文件，如下

  1. apiVersion: elasticsearch.k8s.elastic.co/v1beta1
  2. kind: Elasticsearch
  3. metadata:
  4.   name: es
  5.   namespace: elastic-system
  6. spec:
  7.   version: 7.4.1
  8.   http:
  9.     service:
  10.       metadata:
  11.         creationTimestamp: null
  12.       spec: {}
  13.     tls:
  14.       certificate: {}
  15.       selfSignedCertificate:
  16.         disabled: true
  17.   nodeSets:
  18.   - config:
  19.       node.data: true
  20.       node.ingest: true
  21.       node.master: true
  22.       node.store.allow_mmap: true
  23.       xpack.security.authc.realms:
  24.         native:
  25.           native1:
  26.             order: 1
  27.     count: 1
  28.     name: master
  29.     podTemplate:
  30.       spec:
  31.         containers:
  32.         - env:
  33.           - name: ES_JAVA_OPTS
  34.             value: -Xms1g -Xmx1g
  35.           name: elasticsearch
  36.           resources:
  37.             limits:
  38.               cpu: 2
  39.               memory: 2Gi
  40.             requests:
  41.               cpu: 2
  42.               memory: 2Gi
  43.     volumeClaimTemplates:
  44.     - metadata:
  45.         name: elasticsearch-data
  46.       spec:
  47.         accessModes:
  48.         - ReadWriteMany
  49.         resources:
  50.           requests:
  51.             storage: 10Gi
  52.         storageClassName: managed-nfs-storage

通过NFS做的持久化

可以通过如下命令查看状态

  1. # kubectl get elasticsearches.elasticsearch.k8s.elastic.co -n elastic-system 
  2. NAME   HEALTH   NODES   VERSION   PHASE   AGE
  3. es     green    1       7.4.1     Ready   48m
  4. # kubectl get pod -n elastic-system 
  5. NAME                            READY   STATUS              RESTARTS   AGE
  6. elastic-operator-0              1/1     Running             1          56m
  7. es-es-master-0                  1/1     Running             0          47m

查看svc的信息

  1. # kubectl get svc -n elastic-system
  2. NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
  3. elastic-webhook-server   ClusterIP   10.111.204.184   <none>        443/TCP          57m
  4. es-es-http               ClusterIP   10.100.200.193   <none>        9200/TCP         48m
  5. es-es-master             ClusterIP   None             <none>        <none>           47m
  6. es-es-transport          ClusterIP   None             <none>        9300/TCP         48m

同时是会生成一些secrets，如下：

  1. # kubectl get secrets -n elastic-system
  2. NAME                                   TYPE                                  DATA   AGE
  3. default-token-xjqxq                    kubernetes.io/service-account-token   3      57m
  4. elastic-operator-token-ttckk           kubernetes.io/service-account-token   3      57m
  5. elastic-system-es-kibana-kibana-user   Opaque                                3      23m
  6. elastic-webhook-server-cert            Opaque                                2      57m
  7. es-es-elastic-user                     Opaque                                1      48m
  8. es-es-http-ca-internal                 Opaque                                2      48m
  9. es-es-http-certs-internal              Opaque                                3      48m
  10. es-es-http-certs-public                Opaque                                2      48m
  11. es-es-internal-users                   Opaque                                2      48m
  12. es-es-master-es-config                 Opaque                                1      47m
  13. es-es-remote-ca                        Opaque                                1      48m
  14. es-es-transport-ca-internal            Opaque                                2      48m
  15. es-es-transport-certificates           Opaque                                3      48m
  16. es-es-transport-certs-public           Opaque                                1      48m
  17. es-es-xpack-file-realm                 Opaque                                3      48m

其中quickstart-es-elastic-user就是存放的登录es的密码，我们可以通过以下命令获取明文信息

  1. kubectl get secret -n elastic-system es-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode

然后访问查看ES是否正常（集群内部访问）

  1. curl -k https://elastic:nf963Smr0To2u6AA1dS0u93f@es-es-http:9200

部署kibana

kibana的yaml文件如下：

  1. apiVersion: kibana.k8s.elastic.co/v1beta1
  2. kind: Kibana
  3. metadata:
  4.   name: es-kibana
  5.   namespace: elastic-system
  6. spec:
  7.   count: 1
  8.   elasticsearchRef:
  9.     name: es
  10.   http:
  11.     service:
  12.       metadata:
  13.         creationTimestamp: null
  14.       spec: {}
  15.     tls:
  16.       certificate: {}
  17.       selfSignedCertificate:
  18.         disabled: true
  19.   podTemplate:
  20.     metadata:
  21.       creationTimestamp: null
  22.     spec:
  23.       containers:
  24.       - name: kibana
  25.         resources:
  26.           limits:
  27.             cpu: 1
  28.             memory: 2Gi
  29.           requests:
  30.             cpu: 0.5
  31.             memory: 1Gi
  32.   version: 7.4.1

部署后查看pod状态。

  1. # kubectl get pod -n elastic-system 
  2. NAME                            READY   STATUS              RESTARTS   AGE
  3. elastic-operator-0              1/1     Running             1          58m
  4. es-es-master-0                  1/1     Running             0          48m
  5. es-kibana-kb-5c4468f8bf-msql2   1/1     Running             0          23m

查看svc。

  1. # kubectl get svc -n elastic-system 
  2. NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
  3. elastic-webhook-server   ClusterIP   10.111.204.184   <none>        443/TCP          64m
  4. es-es-http               ClusterIP   10.100.200.193   <none>        9200/TCP         55m
  5. es-es-master             ClusterIP   None             <none>        <none>           55m
  6. es-es-transport          ClusterIP   None             <none>        9300/TCP         55m
  7. es-kibana-kb-http        NodePort    10.105.148.116   <none>        5601:30696/TCP   30m

然后我们用kubectl edit将es-kibana-kb-http 的type类型改为NodePort。
登录kibana，创建index template

  1. PUT /_template/app-template
  2. {
  3.     "index_patterns" : [
  4.       "app-*"
  5.     ],
  6.     "settings" : {
  7.       "index" : {
  8.         "lifecycle" : {
  9.           "name" : "slm-history-ilm-policy"
  10.         },
  11.         "number_of_shards" : "2",
  12.         "number_of_replicas" : "0",
  13.         "highlight.max_analyzed_offset": 10000000
  14.       }
  15.     },
  16.     "mappings" : {
  17.       "dynamic_templates" : [
  18.         {
  19.           "message_field" : {
  20.             "path_match" : "message",
  21.             "mapping" : {
  22.               "norms" : false,
  23.               "type" : "text"
  24.             },
  25.             "match_mapping_type" : "string"
  26.           }
  27.         },
  28.         {
  29.           "string_fields" : {
  30.             "mapping" : {
  31.               "norms" : false,
  32.               "type" : "keyword"
  33.             },
  34.             "match_mapping_type" : "string",
  35.             "match" : "*"
  36.           }
  37.         }
  38.       ]
  39.     },
  40.     "aliases" : { }
  41. }

部署filebeat

配置filebeat的configMap.

  1. apiVersion: v1
  2. data:
  3.   filebeat.yml: |-
  4.     filebeat.inputs:
  5.     - type: log
  6.       fields: {host: '${NODE_NAME}'}
  7.       enabled: true
  8.       paths:
  9.         - /var/logs/webapps/*/*/*-info-*.log
  10.       tail_files: true
  11.       multiline.match: after
  12.       multiline.negate: true
  13.       multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
  14.       processors:
  15.         - dissect:
  16.               tokenizer: "/%{key1}/%{key2}/%{key3}/%{appName}/%{podName}/"
  17.               field: "log.file.path"
  18.               target_prefix: ""
  19.     - type: log
  20.       fields: {host: '${NODE_NAME}'}
  21.       enabled: true
  22.       paths:
  23.         - /var/logs/webapps/*/*.log
  24.       tail_files: true
  25.       multiline.match: after
  26.       multiline.negate: true
  27.       multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}
  28.       processors:
  29.         - dissect:
  30.               tokenizer: "/%{key1}/%{key2}/%{key3}/%{appName}/"
  31.               field: "log.file.path"
  32.               target_prefix: ""
  33.     setup.ilm.enabled: false
  34.     setup.template.pattern: "app-*"
  35.     setup.template.name: "app-template"
  36.     output.elasticsearch:
  37.       pipeline: timestamp-pipeline
  38.       hosts: ["es-es-http:9200"]
  39.       username: elastic
  40.       password: nf963Smr0To2u6AA1dS0u93f 
  41.       index: "app-%{[appName]}-%{+yyyy.MM.dd}"
  42.       bulk_max_size: 100
  43.     processors:
  44.       - rename:
  45.             fields:
  46.               - from: "log.file.path"
  47.                 to: "source"
  48.               - from: "fields.host"
  49.                 to: "node"
  50.             ignore_missing: true
  51.       - drop_fields:
  52.             when:
  53.                 has_fields: ['key1','key2','key3','agent','@metadata','ecs','input','host','log','fields']
  54.             fields: ['key1','key2','key3','agent','@metadata','ecs','input','host','log','fields'] 
  55.     logging.level: error
  56. kind: ConfigMap
  57. metadata:
  58.   labels:
  59.     k8s-app: filebeat
  60.   name: filebeat-config
  61.   namespace: elastic-system

filebeat的yaml如下。

  1. apiVersion: apps/v1
  2. kind: DaemonSet
  3. metadata:
  4.   labels:
  5.     k8s-app: filebeat
  6.   name: filebeat
  7.   namespace: elastic-system
  8. spec:
  9.   revisionHistoryLimit: 10
  10.   selector:
  11.     matchLabels:
  12.       k8s-app: filebeat
  13.   template:
  14.     metadata:
  15.       labels:
  16.         k8s-app: filebeat
  17.     spec:
  18.       containers:
  19.         - args:
  20.             - '-c'
  21.             - /etc/filebeat.yml
  22.             - '-e'
  23.           env:
  24.             - name: NODE_NAME
  25.               valueFrom:
  26.                 fieldRef:
  27.                   apiVersion: v1
  28.                   fieldPath: spec.nodeName
  29.           image: docker.elastic.co/beats/filebeat:7.4.0 
  30.           imagePullPolicy: IfNotPresent
  31.           name: filebeat
  32.           resources:
  33.             limits:
  34.               cpu: 300m
  35.               memory: 300Mi
  36.             requests:
  37.               cpu: 300m
  38.               memory: 300Mi
  39.           securityContext:
  40.             runAsUser: 0
  41.           terminationMessagePath: /dev/termination-log
  42.           terminationMessagePolicy: File
  43.           volumeMounts:
  44.             - mountPath: /etc/filebeat.yml
  45.               name: config
  46.               readOnly: true
  47.               subPath: filebeat.yml
  48.             - mountPath: /usr/share/filebeat/data
  49.               name: data
  50.             - mountPath: /var/logs/webapps
  51.               name: logpath
  52.               readOnly: true
  53.             - mountPath: /var/log
  54.               name: varlog
  55.               readOnly: true
  56.       dnsPolicy: ClusterFirstWithHostNet
  57.       hostNetwork: true
  58.       restartPolicy: Always
  59.       schedulerName: default-scheduler
  60.       securityContext: {}
  61.       terminationGracePeriodSeconds: 30
  62.       volumes:
  63.         - configMap:
  64.             defaultMode: 384
  65.             name: filebeat-config
  66.           name: config
  67.         - hostPath:
  68.             path: /var/lib/filebeat-data
  69.             type: DirectoryOrCreate
  70.           name: data
  71.         - hostPath:
  72.             path: /home/logs
  73.             type: ''
  74.           name: logpath
  75.         - hostPath:
  76.             path: /var/log
  77.             type: ''
  78.           name: varlog

创建模板文件：

  1. PUT /_template/app-template
  2. {
  3.     "index_patterns" : [
  4.       "app-*"
  5.     ],
  6.     "settings" : {
  7.       "index" : {
  8.         "lifecycle" : {
  9.           "name" : "slm-history-ilm-policy"
  10.         },
  11.         "number_of_shards" : "2",
  12.         "number_of_replicas" : "0",
  13.         "highlight.max_analyzed_offset": 10000000
  14.       }
  15.     },
  16.     "mappings" : {
  17.       "dynamic_templates" : [
  18.         {
  19.           "message_field" : {
  20.             "path_match" : "message",
  21.             "mapping" : {
  22.               "norms" : false,
  23.               "type" : "text"
  24.             },
  25.             "match_mapping_type" : "string"
  26.           }
  27.         },
  28.         {
  29.           "string_fields" : {
  30.             "mapping" : {
  31.               "norms" : false,
  32.               "type" : "keyword"
  33.             },
  34.             "match_mapping_type" : "string",
  35.             "match" : "*"
  36.           }
  37.         }
  38.       ]
  39.     },
  40.     "aliases" : { }
  41. }

创建pipeline：

  1. PUT /_ingest/pipeline/timestamp-pipeline
  2. {
  3.     "description" : "fix timestamp",
  4.     "processors" : [
  5.       {
  6.         "grok" : {
  7.           "if" : "! ctx.appName.contains('weipeiapp')",
  8.           "field" : "message",
  9.           "patterns" : [
  10.             "%{TIMESTAMP_ISO8601:timestamp} "
  11.           ]
  12.         },
  13.         "remove" : {
  14.           "if" : "! ctx.appName.contains('weipeiapp')",
  15.           "field" : "@timestamp"
  16.         }
  17.       },
  18.       {
  19.         "date" : {
  20.           "if" : "! ctx.appName.contains('weipeiapp')",
  21.           "field" : "timestamp",
  22.           "timezone" : "Asia/Shanghai",
  23.           "formats" : [
  24.             "yyyy-MM-dd HH:mm:ss.SSS"
  25.           ]
  26.         },
  27.         "remove" : {
  28.           "if" : "! ctx.appName.contains('weipeiapp')",
  29.           "field" : "timestamp"
  30.         }
  31.       }
  32.     ],
  33.     "on_failure" : [
  34.       {
  35.         "set" : {
  36.           "field" : "_index",
  37.           "value" : "{{ _index }}"
  38.         }
  39.       }
  40.     ]
  41.   }