一、环境准备

服务器

Host IP
log-manager 172.17.217.149
log-worker 172.17.118.47

配置Swarm集群

  1. # log-manager
  2. docker swarm init --advertise-addr 172.17.217.149
  4. # log-worker
  5. docker swarm join --token token 172.17.217.149:2377

拉取镜像

  1. # all server
  2. docker pull logstash:7.16.2

目录配置

如使用nfs等网络存储，注意检查挂载权限及存储配置权限，避免踩坑

  1. # all server
  3. # base path
  4. mkdir /data/logstash -p
  5. # log path
  6. mkdir /data/logstash/log
  7. # conf path
  8. mkdir /data/logstash/conf.d
  10. chmod 777 +R /data/logstash

配置文件

logstash.yml

vi /data/logstash/logstash.yml

  1. path.config: /usr/share/logstash/conf.d/*.conf
  2. path.logs: /var/log/logstash

demo1-logstash.conf

接收tcp端口5959，输出至es和stdout

vi /data/logstash/conf.d/logstash.yml

  1. input {
  2.   tcp {
  3.     port => 5959
  4.     mode => "server"
  5.     tags => ["tags"]
  6.     codec => json_lines
  7.   }
  8. }
  9. output {
  10.   elasticsearch {
  11.     hosts => ["127.0.0.1:9200"]
  12.     index => "demo"
  13.   }
  14.   stdout {
  15.     codec => rubydebug
  16.   }
  17. }

/data/logstash/conf.d/demo2-logstash.conf

接收udp端口6666，输出至文件

  1. input {
  2.   udp {
  3.     port                 => 6666
  4.     codec                => cef
  5.     receive_buffer_bytes => 16777216
  6.     workers              => 16
  7.   }
  8. }
  10. output {
  11.   file {
  12.     path => "/var/log/logstash/demo2-%{+YYYY-MM-dd}.log"
  13.     file_mode => 0644
  14.   }
  15. }

二、操作

  1. # log-manager
  2. docker service create \
  3. --replicas 2 \
  4. --name my_log \
  5. --publish published=6666,target=6666,protocol=udp \
  6. --mount type=bind,src=/data/logstash/logstash.yml,dst=/usr/share/logstash/config/logstash.yml \
  7. --mount type=bind,src=/data/logstash/conf.d,dst=/usr/share/logstash/conf.d \
  8. --mount type=bind,src=/data/logstash/log,dst=/var/log/logstash \
  9. logstash:7.16.2

三、测试

logstash-tool.txt

  1. # 将附件工具上传至服务器
  3. # 重命名并授权
  4. mv logstash-tool.txt logstash-tool
  5. chmod +x logstash-tool
  7. # 测试
  8. ./logstash-tool --lmsg=test_msg   --lnet=udp --lport=6666