skywalking监控基于nginx(openresty)+java服务(jenkins)
一、监控nginx服务
1.1、安装配置luajit环境
1.1.1、下载编译安装luajit
wget https://github.com/openresty/luajit2/archive/refs/tags/v2.1-20220411.tar.gz
mkdir /software
tar -xf v2.1-20220411.tar.gz -C /software
mv luajit2-2.1-20220411/ luajit2
cd luajit2
apt-get update && apt -y install make gcc libpcre3 libpcre3-dev openssl libssl-dev zlib1g-dev
make install PREFIX=/usr/local/luajit2-2.1
[root@skywalking-agent-03 luajit2]# make install PREFIX=/usr/local/luajit2-2.1 ==== Building LuaJIT 2.1.0-beta3 ==== make -C src make[1]: Entering directory '/software/luajit2/src' CC lj_init.o HOSTCC host/minilua.o HOSTLINK host/minilua DYNASM host/buildvm_arch.h HOSTCC host/buildvm.o HOSTCC host/buildvm_asm.o HOSTCC host/buildvm_peobj.o HOSTCC host/buildvm_lib.o HOSTCC host/buildvm_fold.o HOSTLINK host/buildvm BUILDVM lj_vm.S ASM lj_vm.o CC lj_assert.o CC lj_gc.o BUILDVM lj_ffdef.h CC lj_err.o CC lj_char.o BUILDVM lj_bcdef.h CC lj_bc.o CC lj_obj.o CC lj_buf.o CC lj_str.o CC lj_tab.o CC lj_func.o CC lj_udata.o CC lj_meta.o CC lj_debug.o CC lj_prng.o CC lj_state.o CC lj_dispatch.o CC lj_vmevent.o CC lj_vmmath.o CC lj_strscan.o CC lj_strfmt.o CC lj_strfmt_num.o CC lj_serialize.o CC lj_api.o CC lj_profile.o CC lj_lex.o CC lj_parse.o CC lj_bcread.o CC lj_bcwrite.o CC lj_load.o CC lj_ir.o CC lj_opt_mem.o BUILDVM lj_folddef.h CC lj_opt_fold.o CC lj_opt_narrow.o CC lj_opt_dce.o CC lj_opt_loop.o CC lj_opt_split.o CC lj_opt_sink.o CC lj_mcode.o CC lj_snap.o CC lj_record.o CC lj_crecord.o BUILDVM lj_recdef.h CC lj_ffrecord.o CC lj_asm.o CC lj_trace.o CC lj_gdbjit.o CC lj_ctype.o CC lj_cdata.o CC lj_cconv.o CC lj_ccall.o CC lj_ccallback.o CC lj_carith.o CC lj_clib.o CC lj_cparse.o CC lj_lib.o CC lj_alloc.o CC lib_aux.o BUILDVM lj_libdef.h CC lib_base.o CC lib_math.o CC lib_bit.o CC lib_string.o CC lib_table.o CC lib_io.o CC lib_os.o CC lib_package.o CC lib_debug.o CC lib_jit.o CC lib_ffi.o CC lib_buffer.o CC lib_init.o CC lj_str_hash.o AR libluajit.a CC luajit.o BUILDVM jit/vmdef.lua DYNLINK libluajit.so LINK luajit OK Successfully built LuaJIT make[1]: Leaving directory '/software/luajit2/src' ==== Successfully built LuaJIT 2.1.0-beta3 ==== ==== Installing LuaJIT 2.1.0-beta3 to /usr/local/luajit2-2.1 ==== mkdir -p /usr/local/luajit2-2.1/bin /usr/local/luajit2-2.1/lib /usr/local/luajit2-2.1/include/luajit-2.1 /usr/local/luajit2-2.1/share/man/man1 /usr/local/luajit2-2.1/lib/pkgconfig /usr/local/luajit2-2.1/share/luajit-2.1.0-beta3/jit /usr/local/luajit2-2.1/share/lua/5.1 /usr/local/luajit2-2.1/lib/lua/5.1 cd src && install -m 0755 luajit /usr/local/luajit2-2.1/bin/luajit-2.1.0-beta3 cd src && test -f libluajit.a && install -m 0644 libluajit.a /usr/local/luajit2-2.1/lib/libluajit-5.1.a || : rm -f /usr/local/luajit2-2.1/lib/libluajit-5.1.so.2.1.0 /usr/local/luajit2-2.1/lib/libluajit-5.1.so /usr/local/luajit2-2.1/lib/libluajit-5.1.so.2 cd src && test -f libluajit.so && \ install -m 0755 libluajit.so /usr/local/luajit2-2.1/lib/libluajit-5.1.so.2.1.0 && \ ( ldconfig -n 2>/dev/null /usr/local/luajit2-2.1/lib || : ) && \ ln -sf libluajit-5.1.so.2.1.0 /usr/local/luajit2-2.1/lib/libluajit-5.1.so && \ ln -sf libluajit-5.1.so.2.1.0 /usr/local/luajit2-2.1/lib/libluajit-5.1.so.2 || : cd etc && install -m 0644 luajit.1 /usr/local/luajit2-2.1/share/man/man1 cd etc && sed -e "s|^prefix=.*|prefix=/usr/local/luajit2-2.1|" -e "s|^multilib=.*|multilib=lib|" luajit.pc > luajit.pc.tmp && \ install -m 0644 luajit.pc.tmp /usr/local/luajit2-2.1/lib/pkgconfig/luajit.pc && \ rm -f luajit.pc.tmp cd src && install -m 0644 lua.h lualib.h lauxlib.h luaconf.h lua.hpp luajit.h /usr/local/luajit2-2.1/include/luajit-2.1 cd src/jit && install -m 0644 bc.lua bcsave.lua dump.lua p.lua v.lua zone.lua dis_x86.lua dis_x64.lua dis_arm.lua dis_arm64.lua dis_arm64be.lua dis_ppc.lua dis_mips.lua dis_mipsel.lua dis_mips64.lua dis_mips64el.lua vmdef.lua /usr/local/luajit2-2.1/share/luajit-2.1.0-beta3/jit ln -sf luajit-2.1.0-beta3 /usr/local/luajit2-2.1/bin/luajit ==== Successfully installed LuaJIT 2.1.0-beta3 to /usr/local/luajit2-2.1 ==== [root@skywalking-agent-03 luajit2]#
1.1.2、配置系统环境变量，方便编译安装nginx使用
[root@skywalking-agent-03 ~]# vim /etc/profile
export LUAJIT_LIB=/usr/local/luajit2-2.1/lib
export LUAJIT_INC=/usr/local/luajit2-2.1/include/luajit-2.1
[root@skywalking-agent-03 ~]# source /etc/profile
1.1.3、加载luajit2模块，nginx启动需要调用libluajit-5.1.so.2模块
[root@skywalking-agent-03 ~]# vim /etc/ld.so.conf.d/libc.conf
# libc default configuration /usr/local/lib /usr/local/luajit2-2.1/lib/
[root@skywalking-agent-03 ~]# ldconfig
1.2、编译安装lua核心库，nginx需要加载lua库
1.2.1、准备lua-resty-core
[root@skywalking-agent-03 software]# wget wget https://github.com/openresty/lua-resty-core/archive/refs/tags/v0.1.23.tar.gz
[root@skywalking-agent-03 software]# tar -xf lua-resty-core-0.1.23.tar.gz
[root@skywalking-agent-03 software]# cd lua-resty-core-0.1.23/
[root@skywalking-agent-03 lua-resty-core-0.1.23]# make install PREFIX=/usr/local/luacore
install -d /usr/local/luacore/lib/lua//resty/core/ install -d /usr/local/luacore/lib/lua//ngx/ install -d /usr/local/luacore/lib/lua//ngx/ssl install lib/resty/*.lua /usr/local/luacore/lib/lua//resty/ install lib/resty/core/*.lua /usr/local/luacore/lib/lua//resty/core/ install lib/ngx/*.lua /usr/local/luacore/lib/lua//ngx/ install lib/ngx/ssl/*.lua /usr/local/luacore/lib/lua//ngx/ssl/
1.2.2、准备lua-resty-lrucache
[root@skywalking-agent-03 ~]# wget https://github.com/openresty/lua-resty-lrucache/archive/refs/tags/v0.13.tar.gz
[root@skywalking-agent-03 ~]# tar -xf lua-resty-lrucache-0.13.tar.gz -C /software
[root@skywalking-agent-03 ~]# cd /software/lua-resty-lrucache-0.13/
[root@skywalking-agent-03 lua-resty-lrucache-0.13]# make install PREFIX=/usr/local/luacore
install -d //usr/local/luacore/lib/lua//resty/lrucache install lib/resty/*.lua //usr/local/luacore/lib/lua//resty/ install lib/resty/lrucache/*.lua //usr/local/luacore/lib/lua//resty/lrucache/
[root@skywalking-agent-03 lua-resty-lrucache-0.13]#
1.2.3、编译安装lua-cjson
[root@skywalking-agent-03 ~]# wget https://github.com/openresty/lua-cjson/archive/refs/tags/2.1.0.tar.gz
[root@skywalking-agent-03 ~]# tar -xf 2.1.0.tar.gz -C /software
[root@skywalking-agent-03 ~]# cd /software/
[root@skywalking-agent-03 software]# cd lua-cjson-2.1.0/
[root@skywalking-agent-03 lua-cjson-2.1.0]# vim Makefile
20 CJSON_LDFLAGS = -shared 21 LUA_INCLUDE_DIR = $(PREFIX)/include 22 LUA_CMODULE_DIR = $(PREFIX)/lib/lua/$(LUA_VERSION) 修改为: 20 CJSON_LDFLAGS = -shared 21 LUA_INCLUDE_DIR = /usr/local/luajit2-2.1/include/luajit-2.1 22 LUA_CMODULE_DIR = $(PREFIX)/lib/lua/$(LUA_VERSION)
[root@skywalking-agent-03 lua-cjson-2.1.0]# vim lua_cjson.c
1291 /* ===== INITIALISATION ===== */ 1292 1293 #if !defined(LUA_VERSION_NUM) || LUA_VERSION_NUM < 502 1294 /* Compatibility for Lua 5.1. 1295 * 1296 * luaL_setfuncs() is used to create a module table where the functions have 1297 * json_config_t as their first upvalue. Code borrowed from Lua 5.2 source. */ 1298 static void luaL_setfuncs (lua_State *l, const luaL_Reg *reg, int nup) 1299 { 改成： 1291 /* ===== INITIALISATION ===== */ 1292 1293 #if !defined(LUA_VERSION_NUM) || LUA_VERSION_NUM < 502 1294 /* Compatibility for Lua 5.1. 1295 * 1296 * luaL_setfuncs() is used to create a module table where the functions have 1297 * json_config_t as their first upvalue. Code borrowed from Lua 5.2 source. */ 1298 void luaL_setfuncs (lua_State *l, const luaL_Reg *reg, int nup) 1299 {
[root@skywalking-agent-03 lua-cjson-2.1.0]# make
cc -c -O3 -Wall -pedantic -DNDEBUG -I/usr/local/luajit2-2.1/include/luajit-2.1 -fpic -o lua_cjson.o lua_cjson.c In file included from lua_cjson.c:47: fpconv.h:15:20: warning: inline function ‘fpconv_init’ declared but never defined 15 | extern inline void fpconv_init(); | ^~~~~~~~~~~ cc -c -O3 -Wall -pedantic -DNDEBUG -I/usr/local/luajit2-2.1/include/luajit-2.1 -fpic -o strbuf.o strbuf.c cc -c -O3 -Wall -pedantic -DNDEBUG -I/usr/local/luajit2-2.1/include/luajit-2.1 -fpic -o fpconv.o fpconv.c cc -shared -o cjson.so lua_cjson.o strbuf.o fpconv.o
[root@skywalking-agent-03 lua-cjson-2.1.0]# make install
mkdir -p //usr/local/lib/lua/5.1 cp cjson.so //usr/local/lib/lua/5.1 chmod 755 //usr/local/lib/lua/5.1/cjson.so
1.2.4、准备nginx_devel_kit源码
[root@skywalking-agent-03 ~]# wget https://github.com/vision5/ngx_devel_kit/archive/refs/tags/v0.3.1.tar.gz
[root@skywalking-agent-03 ~]# tar -xf v0.3.1.tar.gz -C /software/
1.2.5、准备lua-nginx-module源码
[root@skywalking-agent-03 ~]# wget https://github.com/openresty/lua-nginx-module/archive/refs/tags/v0.10.21.tar.gz
[root@skywalking-agent-03 ~]# tar -xf v0.10.21.tar.gz -C /software/
二、编译安装nginx
[root@skywalking-agent-03 ~]# wget https://nginx.org/download/nginx-1.20.2.tar.gz
[root@skywalking-agent-03 ~]# tar -xf nginx-1.20.2.tar.gz -C /software/
[root@skywalking-agent-03 software]# cd nginx-1.20.2/
./configure --prefix=/apps/nginx \ --with-http_ssl_module \ --with-http_v2_module \ --with-http_realip_module \ --with-http_stub_status_module \ --with-http_gzip_static_module \ --with-pcre \ --with-stream \ --with-stream_ssl_module \ --with-stream_realip_module \ --add-module=/software/ngx_devel_kit-0.3.1/ \ --add-module=/software/lua-nginx-module-0.10.21/
[root@skywalking-agent-03 nginx-1.20.2]# make && make install
验证lua环境
[root@skywalking-agent-03 ~]# egrep -v "^$|^[[:space:]]+#|^#" /apps/nginx/conf/nginx.conf
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; lua_package_path "/usr/local/luacore/lib/lua/?.lua;;"; sendfile on; keepalive_timeout 65; server { listen 80; server_name localhost; location / { root html; index index.html index.htm; } location /hello { default_type text/html; content_by_lua_block { ngx.say("Hello Lua!") } } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
[root@skywalking-agent-03 ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@skywalking-agent-03 ~]#
[root@skywalking-agent-03 ~]# /apps/nginx/sbin/nginx
[root@skywalking-agent-03 ~]# netstat -tnlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 672/systemd-resolve tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 830/sshd: /usr/sbin tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 8998/sshd: root@pts tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 14361/nginx: master tcp6 0 0 :::22 :::* LISTEN 830/sshd: /usr/sbin tcp6 0 0 ::1:6010 :::* LISTEN 8998/sshd: root@pts [root@skywalking-agent-03 ~]#
三、配置skywalking探针
[root@skywalking-agent-03 data]# wget https://github.com/apache/skywalking-nginx-lua/archive/refs/tags/v0.6.0.tar.gz
[root@skywalking-agent-03 data]# tar -xf v0.6.0.tar.gz
[root@skywalking-agent-03 ~]# vim /apps/nginx/conf/nginx.conf
[root@skywalking-agent-03 ~]# egrep -v "^$|^[[:space:]]+#|^#" /apps/nginx/conf/nginx.conf
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; lua_package_path "/usr/local/luacore/lib/lua/?.lua;/data/skywalking-nginx-lua-0.6.0/lib/?.lua;;"; sendfile on; keepalive_timeout 65; server { listen 80; server_name localhost; location / { root html; index index.html index.htm; } location /hello { default_type text/html; content_by_lua_block { ngx.say("Hello Lua!") } } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } include /apps/nginx/conf/conf.d/*.conf; }
[root@skywalking-agent-03 ~]# mkdir -p /apps/nginx/conf/conf.d/
[root@skywalking-agent-03 ~]# cd /apps/nginx/conf/conf.d
[root@skywalking-agent-03 conf.d]# vim myserver.conf
#lua_package_path "/data/skywalking-nginx-lua-0.6.0/lib/?.lua;;"; lua_shared_dict tracing_buffer 100m; init_worker_by_lua_block { local metadata_buffer = ngx.shared.tracing_buffer metadata_buffer:set('serviceName', 'myserver-nginx') metadata_buffer:set('serviceInstanceName', 'skywalking-agent-03') metadata_buffer:set('includeHostInEntrySpan', false) require("skywalking.util").set_randomseed() require("skywalking.client"):startBackendTimer("http://172.16.88.187:12800") skywalking_tracer = require("skywalking.tracer") } server { listen 80; server_name www.myserver.com; location /jenkins { default_type text/html; rewrite_by_lua_block { skywalking_tracer:start("www.myserver.com") } proxy_pass http://172.16.88.189:8080; body_filter_by_lua_block { if ngx.arg[2] then skywalking_tracer:finish() end } log_by_lua_block { skywalking_tracer:prepareForReport() } } location / { default_type test/html; root html; rewrite_by_lua_block { skywalking_tracer:start("local") } body_filter_by_lua_block { if ngx.arg[2] then skywalking_tracer:finish() end } log_by_lua_block { skywalking_tracer:prepareForReport() } } }
[root@skywalking-agent-03 ~]# vim /apps/nginx/html/index.html
[root@skywalking-agent-03 ~]# cat /apps/nginx/html/index.html
skywalking monitor nginx test !!!!
[root@skywalking-agent-03 ~]#
[root@skywalking-agent-03 ~]# /apps/nginx/sbin/nginx -t
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
[root@skywalking-agent-03 ~]# /apps/nginx/sbin/nginx -s reload
[root@skywalking-agent-03 ~]#
配置本地hosts解析
此时访问报错，查看日志缺少tablepool库
[root@skywalking-agent-03 ~]# less /apps/nginx/logs/error.log
解决办法：
[root@skywalking-agent-03 ~]# git clone https://github.com/openresty/lua-tablepool.git
[root@skywalking-agent-03 ~]# cd lua-tablepool-master/
[root@skywalking-agent-03 lua-tablepool-master]# cp lib/tablepool.lua /data/skywalking-nginx-lua-0.6.0/lib/
三、 验证skywalking server端
