前提:有可用的kubernetes集群和skywalking监控。
软件版本信息:
| 软件 | 版本 |
|---|---|
| kubernetes | 1.17.2 |
| nginx-ingress-controller | 0.34.1 |
| skywalking | 8.1.0 |
| skywalking-nginx-lua | 0.2.0 |
下面直接进入正题……
(1)下载skywalking-nginx-lua
git clone https://github.com/apache/skywalking-nginx-lua.git
(2)修改util.lua文件名,因为其和nginx-ingress默认的一个lua脚本名字冲突,这里改一下。
$ skywalking-nginx-lua/lib/skywalking# 修改文件名$ mv util.lua swutil.lua# 修改文件调用## 可以使用grep查看一下哪些文件有调用这个lua$ grep util `find ./ -type f`./correlation_context.lua:local Util = require('util')./segment_ref.lua:local Util = require('util')./span.lua:local Util = require('util')./tracing_context.lua:local Util = require('util')./swutil_test.lua:local Util = require('util')# 将里面调用的util都改为swuitl
(3)修改Nginx-ingress的nginx.tmpl模板文件,增加Skywalking的配置。
- 添加Skywalking Lua脚本扫描路径
- 增加环境变量读取,如:SW_SERVICE_NAME、SW_SERVICE_INSTANCE_NAME、SW_BACKEND_SERVERS
- 添加Tracing使用的缓存
tracing_buffer - 设置Skywalking Lua Agent的初始化方法,并将相关配置从环境变量中提取。
- 设置http节点的追踪配置。
```shell
Skywalking ENV
env SW_SERVICE_NAME; env SW_SERVICE_INSTANCE_NAME; env SW_BACKEND_SERVERS;
events { multi_accept {{ if $cfg.EnableMultiAccept }}on{{ else }}off{{ end }}; worker_connections {{ $cfg.MaxWorkerConnections }}; use epoll; }
http {
# 引入lua脚本lua_package_path "/etc/nginx/lua/?.lua;/etc/nginx/lua/skywalking/?.lua;;";# 使用缓存lua_shared_dict tracing_buffer 100m;{{ buildLuaSharedDictionaries $cfg $servers }}init_by_lua_block {collectgarbage("collect")-- init moduleslocal ok, resok, res = pcall(require, "lua_ingress")if not ok thenerror("require failed: " .. tostring(res))elselua_ingress = reslua_ingress.set_config({{ configForLua $all }})endok, res = pcall(require, "configuration")if not ok thenerror("require failed: " .. tostring(res))elseconfiguration = resendok, res = pcall(require, "balancer")if not ok thenerror("require failed: " .. tostring(res))elsebalancer = resend{{ if $all.EnableMetrics }}ok, res = pcall(require, "monitor")if not ok thenerror("require failed: " .. tostring(res))elsemonitor = resend{{ end }}ok, res = pcall(require, "certificate")if not ok thenerror("require failed: " .. tostring(res))elsecertificate = rescertificate.is_ocsp_stapling_enabled = {{ $cfg.EnableOCSP }}endok, res = pcall(require, "plugins")if not ok thenerror("require failed: " .. tostring(res))elseplugins = resend-- load all plugins that'll be used hereplugins.init({ {{ range $idx, $plugin := $cfg.Plugins }}{{ if $idx }},{{ end }}{{ $plugin | quote }}{{ end }} })}init_worker_by_lua_block {
……
-- Skywalkinglocal metadata_buffer = ngx.shared.tracing_buffermetadata_buffer:set('serviceName', os.getenv("SW_SERVICE_NAME"))metadata_buffer:set('serviceInstanceName', os.getenv("SW_SERVICE_INSTANCE_NAME"))-- set random seedrequire("swutil").set_randomseed()require("client"):startBackendTimer(os.getenv("SW_BACKEND_SERVERS"))}
……
rewrite_by_lua_block {lua_ingress.rewrite({{ locationConfigForLua $location $all }})balancer.rewrite()plugins.run()-- Skywalkingrequire("tracer"):start({{ buildUpstreamName $location | quote }})}
……
body_filter_by_lua_block {-- Skywalkingif ngx.arg[2] thenrequire("tracer"):finish()end}
……
log_by_lua_block {balancer.log(){{ if $all.EnableMetrics }}monitor.call(){{ end }}plugins.run()-- Skywalkingrequire("tracer"):prepareForReport()}
…….
(4)将这些脚本添加到nginx-ingress镜像中,Dockerfile如下:```dockerfileFROM quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0ADD --chown=www-data nginx.tmpl /etc/nginx/templateADD --chown=www-data skywalking /etc/nginx/lua/skywalking
(5)构建镜像并上传仓库
$ docker build -t registry.cn-hangzhou.aliyuncs.com/rookieops/ingress-nginx-controller-skywalking:0.34.1 .$ docker push registry.cn-hangzhou.aliyuncs.com/rookieops/ingress-nginx-controller-skywalking:0.34.1
(6)修改nginx-ingress的deployment文件,主要增加以下环境变量
......containers:- name: controllerimage: registry.cn-hangzhou.aliyuncs.com/rookieops/nginx-ingress-controller:0.32.0imagePullPolicy: IfNotPresent......- name: SW_SERVICE_NAMEvalue: Kubernetes Ingress- name: SW_BACKEND_SERVERSvalue: http://skywalking-oap.skywalking.svc.cluster.local:12800- name: SW_SERVICE_INSTANCE_NAMEvalueFrom:fieldRef:fieldPath: metadata.uid......
(7)然后重新部署ingress-controller应用。
然后可以在skywalking的面板上看到了。
已将所需的代码都放在github了,仓库地址:https://github.com/sunsharing-note/skywalking-ingress.git
参考:
