Nginx实现四层代理(五)
1.L4 vs L7
Nginx/Tengine | 四层负载 | 七层负载 |
---|---|---|
共同点 | 转发与处理数据 | 转发与处理数据 |
工作层次不同 | 传输层(tcp、udp) | 应用层/https、请求URI |
应用场景不同 | 端口,非web服务器使用,比如数据库,或者某个应用。 | 用于大部分web服务 http/https协议,根据域名、uri等等 |
效率 | 如果使用四层并且追求性能,推荐lvs即可 | 七层相对四层来说,开销还要更大一些 |
Tips:大部分七层负载即可,追求性能可以考虑四层(LVS DR模式)
2. 环境准备
主机 | IP | ||
---|---|---|---|
web01 | 172.16.1.7 | 创建端口8848 | nc -kl 8848 |
web02 | 172.16.1.8 | 创建端口8848 | nc -kl 8848 |
lb01 | 10.0.0.5 | 进行转发8888 | stream模块 |
2.1 下载nginx源码包
之前lb01就已经通过yum安装过nginx,但是默认不支持stream这个模块,需要重新编译。版本如下
#查看nginx版本
[root@lb01 ~]# nginx -v
nginx version: nginx/1.20.1
#查看nginx的详细编译信息
[root@lb01 ~]# nginx -V
nginx version: nginx/1.20.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)
built with OpenSSL 1.1.1k FIPS 25 Mar 2021
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'
#下载源码包
[root@lb01 ~]# mkdir -p /server/soft
[root@lb01 ~]# wget -O /server/soft/nginx.tar.gz http://nginx.org/download/nginx-1.20.1.tar.gz
2.2 备份原nginx文件
[root@lb01 ~]# mv /usr/sbin/nginx /usr/sbin/nginx.bak
[root@lb01 ~]# cp -r /etc/nginx{,.bak}
2.3 重新编译nginx
在源码包目录下检查模块是否支持,比如这次添加 stream 模块
[root@lb01 ~]# cd /server/soft/nginx-1.20.1/
[root@lb01 nginx-1.20.1]# ./configure --help | grep with-stream
--with-stream enable TCP/UDP proxy module
--with-stream=dynamic enable dynamic TCP/UDP proxy module
--with-stream_ssl_module enable ngx_stream_ssl_module
--with-stream_realip_module enable ngx_stream_realip_module
--with-stream_geoip_module enable ngx_stream_geoip_module
--with-stream_geoip_module=dynamic enable dynamic ngx_stream_geoip_module
--with-stream_ssl_preread_module enable ngx_stream_ssl_preread_module
Tips:disable表示已有该模块,编译时不需要添加。enable表示不支持,需要编译时自己添加!
根据前面查到的已有的模块,加上本次需新增的模块--with-stream
[root@lb01 nginx-1.20.1]# ./configure --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' --with-stream
Tips:make完成后不要继续输入make install,以免现在的nginx出现问题!
以上编译时,如出现缺少依赖一般需要安装以下模块,安装完再次编译:
yum -y install libxml2 libxml2-dev libxslt-devel
yum -y install gd-devel
yum -y install perl-devel perl-ExtUtils-Embed
yum -y install GeoIP GeoIP-devel GeoIP-data
yum -y install pcre-devel
yum -y install openssl openssl-devel
yum -y install gcc
yum -y install redhat-rpm-config
yum -y install gperftools
yum -y install make zlib zlib-devel gcc-c++ libtool openssl openssl-devel
先验证一下
[root@lb01 nginx-1.20.1]# objs/nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@lb01 nginx-1.20.1]# objs/nginx -V
nginx version: nginx/1.20.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' --with-stream
2.4 替换nginx并重启
进行替换操作并重启nginx,最后通过reload检查模块参数(看最后)
[root@lb01 nginx-1.20.1]# cp objs/nginx /usr/sbin/
[root@lb01 nginx-1.20.1]# systemctl reload nginx
[root@lb01 nginx-1.20.1]# nginx -V
nginx version: nginx/1.20.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)
built with OpenSSL 1.0.2k-fips 26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E' --with-stream
3. 配置stream模块
lb01下进行(stream与http平级)
[root@lb01 ~]# vim /etc/nginx/nginx.conf
}
stream {
upstream l4_pools {
server 172.16.1.7:8848;
server 172.16.1.8:8848;
}
server {
listen 8888;
proxy_pass l4_pools;
}
}
重新加载配置文件
[root@lb01 ~]# systemctl reload nginx
结束之前的nginx进程
[root@lb01 ~]# pkill nginx
查看端口状态
[root@lb01 ~]# ss -lntup | grep 8888
tcp LISTEN 0 128 *:8888 *:* users:(("nginx",pid=86056,fd=8),("nginx",pid=86055,fd=8))
4. 测试
ms01主机上通过连接lb01主机上的8888端口,经过lb01主机上nginx配置的strerm模块进行转发到后端节点(web01、web02)上,实现四层转发。
作者:YinJay
Email:szgetshell@163.com