网络,防火墙,nginx点滴

记录一下最近的笔记

redis的一些坑,老外总结得不错
比如使用scan/cursor方式来拉数据来替代keys命令
http://www.programmersought.com/article/89521650540/
https://redis.io/commands/scan
能用:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    public Set<String> scan(String matchKey) {
        Set<String> keys = redisTemplate.execute((RedisCallback<Set<String>>) connection -> {
            Set<String> keysTmp = new HashSet<>();
            Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match("*" + matchKey + "*").count(1000).build());
            while (cursor.hasNext()) {
                keysTmp.add(new String(cursor.next()));
            }
            return keysTmp;
        });
 
        return keys;
    }
我的代码成品, 对count的理解还是不深,待挖掘:
    public Set<String> scanKeys(String keyPattern) {
        Set<String> keys = (Set<String>) redisTemplate.execute((RedisCallback<Set<String>>) connection -> {
            Set<String> keysTmp = new HashSet<>();
            ScanOptions scanOptions = new ScanOptions.ScanOptionsBuilder().match(keyPattern).build();
            Cursor<byte[]> cursor = connection.scan(scanOptions);
            while (cursor.hasNext()) {
                keysTmp.add(new String(cursor.next()));
            }
            return keysTmp;
        });
        return keys;
    }
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>



主要还是安装es7的问题.
https://blog.csdn.net/u011265001/article/details/100084335?depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1&utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1

NG开启basic 认证
官方的文档
https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
老外写的, 不错~~~
https://www.tecmint.com/setup-nginx-basic-http-authentication/

yum install nginx, --->安装在了 /usr/sbin/nginx, 配置文件在/etc/nginx/中

htpasswd /etc/nginx/.htpasswd kibana
tips: 这个工具的安装
# yum install httpd-tools        [RHEL/CentOS]
$ sudo apt install apache2-utils    [Debian/Ubuntu]


然后交互式,输入密码, 就生成了密码文件
$ cat /xxxx/.htpasswd
user1:$apr1$/woC1jnP$KAh0SsVn5qeSMjTtn0E9Q0

配置在nginx的配置文件中
location /api {
    auth_basic           “Administrator’s Area”;
    auth_basic_user_file /etc/apache2/.htpasswd; 
}
就OK了.
https://www.cnblogs.com/silent2012/p/8377837.html 可以参考


官网reverse proxy的配置在这里sample
https://nginx.org/en/docs/http/ngx_http_proxy_module.html?_ga=2.27153012.815603832.1587469798-958214196.1587469798#example


############################################################################

docker问题解决, 同下:
https://www.cnblogs.com/elson-zeng/p/12553329.html
CentOS7 firewalld docker 端口映射问题，firewall开放端口后，还是不能访问，解决方案
# 宿主机ip: 192.168.91.19
docker run -itd --name tomcat -p 8080:8080 tomcat /usr/local/apache-tomcat-9.0.30/bin/startup.sh

# 防火墙放开8080端口
firewall-cmd --add-port=8080/tcp --permanent

# 问题：发现访问：192.168.91.19:8080 访问不通，关闭firewall后，又可以访问通了
# 解决方案，把docker0网卡添加到trusted域
firewall-cmd --permanent --zone=trusted --change-interface=br-d2aa50162455
# 重启加载配置
firewall-cmd --reload
# firewall-cmd相关命令：https://www.cnblogs.com/Raodi/p/11625487.html

我的问题是, 命名防火墙禁用掉了9200端口, 别的机器还是可以访问到.
解决方法是,
https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file
#The default location of the configuration file on Linux is /etc/docker/daemon.json. 
#The --config-file flag can be used to specify a non-default location.
docker的daemon.json里配置:
"iptables": false   --->把iptable禁用掉,--->直接导致,kibana无法访问elasticsearch了
然后再按上面博客, 把docker的一个区域加入了trusted里. --->kibana可以访问elasticsearch了.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
看来碰到问题的人好多
#跟我差不多
https://www.jianshu.com/p/69d3ab177655
https://www.jianshu.com/p/10c467600ef9



#docker service的配置文件在/usr/lib/systemd/system/docker.service
#当时配置docker服务的代理 在这里 /etc/systemd/system/docker.service.d/http-proxy.conf

这里涉及到了3个配置文件:
/etc/docker/daemon.json                 #守护进程各种配置, 配置项非常多, 包括docker镜像站点等
/usr/lib/systemd/system/docker.service  #服务配置文件
/etc/systemd/system/docker.service.d/http-proxy.conf   #配置代理


所以, 需要学习很多相关的知识, 算是还债. 因为一直没有搞得非常通透, 导致解决问题磕磕绊绊
1.systemctl命令相关, 启动脚本在哪里? 配置文件在哪里
2.systemctl daemon相关, 配置文件在哪里, 怎么配的

https://www.cnblogs.com/brucewhite/p/11608853.html
systemctl --help  查看帮助!!!!!!!
systemctl  enable  |  disable |  is-enabled  | status  | is-active  unit
systemctl   get-default|set-default graphical.target|multi-user.target islate 在线切换模式  
systemctl reload-daemon  加载新的unit 配置文件  
systemd的unit 的配置文件位置:redhat系列： 
目录:/usr/lib/systemd/system/ 下 
enable 是在 /etc/systemd/system/multi-user.target.wants/  这个目录下 做 unit 配置文件的软链：

3.linux自动启动的几种方式, init.d, chkconfig, service等等几个概念
https://www.cnblogs.com/liuxia912/p/10960610.html
#CentOS 7的服务systemctl脚本存放在：/usr/lib/systemd/，有系统（system）和用户（user）之分，
#像需要开机不登陆就能运行的程序，还是存在系统服务里吧，即：/usr/lib/systemd/system目录下
#对于那些支持 Systemd 的软件，安装的时候，会自动在/usr/lib/systemd/system目录添加一个配置文件。

在配置vnc服务器的时候也碰到了这个
systemctl daemon-reload

>>待续

4.防火墙的几种配置, firewalld和iptables 

>>>>>>>>>>>>>>>>>>>>>>>>>>>
$ firewall-cmd --zone=public --remove-port=10050/tcp
##这一句 runtime-to_permanent
$ firewall-cmd --runtime-to-permanent
$ firewall-cmd --reload 
>>>>>>>>>>>>>>>>>>>>>>>>>>>


https://www.tecmint.com/start-stop-disable-enable-firewalld-iptables-firewall/
下面几个课题有空再看
Understanding IPtables Firewall Basics and Tips
Configure Iptables Firewall in Linux
Configure FirewallD in Linux
Useful FirewallD Rules to Manage Firewall in Linux
How to Control Network Traffic Using FirewallD and Iptables

# Also, you can mask the firewall service 
# which creates a symbolic link of the firewall.service to /dev/null, 
# thus disabling the service.
# This is reverse of masking the service. 
# This removes the symlink of the service created during masking, 
# thus re-enabling the service.
systemctl statrt/stop/restart/disable/enable/mask/unmask/status firewalld
firewall-cmd --state


systemctl restart firewalld.service(为什么要加一个.service, 是什么意思?)
systemctl is-enabled firewalld.service          #查看服务是否开机启动
systemctl list-unit-files|grep enabled          #查看已启动的服务列表
systemctl --failed                              #查看启动失败的服务列表  

关于iptables
#How to Start/Stop and Enable/Disable IPtables Service
systemctl start/stop/restart/disable/enable/status iptables
service iptables start/stop/save/status

#ubuntu/debian?
sudo ufw enable/disable/status

iptables -L -n -v

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
关于firewalld, 这个老外写得太好了!
https://www.liquidweb.com/kb/an-introduction-to-firewalld/

firewall-cmd --permanent --add-port=22/TCP
firewall-cmd --permanent --add-port=53/UDP
firewall-cmd --permanent --remove-port=444/tcp
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --remove-service=mysql

#whitelist an ip address
firewall-cmd --permanent --add-source=192.168.1.100
#以CIDR地址形式加一个范围, 这是最近学到的.
firewall-cmd --permanent --add-source=192.168.1.0/24
firewall-cmd --permanent --remove-source=192.168.1.100

Block an IP Address
#As the firewall-cmd tool is mostly used for opening or allowing access, 
#rich rules are needed to block an IP. 
#Rich rules are similar in form to the way iptables rules are written.
firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.100' reject"
#CIDR range
firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.0/24' reject"

Whitelist an IP Address for a Specific Port (More Rich Rules)
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'
firewall-cmd --permanent --remove-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'

#save
firewall-cmd --reload
firewall-cmd --runtime-to-permanent
#show
firewall-cmd --list-all

Bonus #1 :: Firewalld GUI Configuration 
You can find these under Applications > Sundry menu on CentOS 7.x and RedHat 7.x servers that have graphical access to. 
To install this application, from the command line, you can run the command:
sudo yum install firewall-config

Bonus #2 :: Firewalld Direct Rules
/etc/firewalld/direct.xml

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
man 1 systemctl
SYSTEMCTL(1)                  systemctl                 SYSTEMCTL(1)

NAME
       systemctl - Control the systemd system and service manager

SYNOPSIS
       systemctl [OPTIONS...] COMMAND [NAME...]

DESCRIPTION
       systemctl may be used to introspect and control the state of
       the "systemd" system and service manager. Please refer to
       systemd(1) for an introduction into the basic concepts and
       functionality this tool manages.

>>>加固Linux
https://www.liquidweb.com/kb/security-for-your-linux-server/

>>>>>
中文的在这里, 比较冗长, 没有好好排版, 内容不错.
https://www.linuxidc.com/Linux/2018-01/150072.htm
firewall daemon 独立于 system-config-firewall，但二者不能同时使用。
如果你想使用自己的 iptables 和 ip6tables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ，
启用 iptables 和ip6tables:
yum install iptables-services
systemctl mask firewalld.service
systemctl enable iptables.service
systemctl enable ip6tables.service

静态防火墙规则配置文件是 /etc/sysconfig/iptables 以及 /etc/sysconfig/ip6tables .

什么是区域？
网络区域定义了网络连接的可信等级。这是一个一对多的关系，这意味着一次连接可以仅仅是一个区域的一部分，
而一个区域可以用于很多连接。

>>>
https://www.cnblogs.com/Raodi/p/11625487.html#_label1

>>>>>>>>>>>>>>>>>>>>>>>>
若 Nginx 收到请求为 https://ngxin_server_name/hello/world
而 Nginx 代理的路径为 /hello/（即在 location /hello/ 内设置代理）
则不以 / 结尾的被代理服务器收到的请求路径是 /hello/world
以 / 结尾的被代理服务器收到的请求路径是 /world
如果是为了在同一个域名下以不同路径分配不同的APP应选择后者以 / 结尾
原文链接：https://blog.csdn.net/randomparty/java/article/details/80961189

 

记录一下最近的笔记
redis的一些坑,老外总结得不错比如使用scan/cursor方式来拉数据来替代keys命令http://www.programmersought.com/article/89521650540/https://redis.io/commands/scan能用:>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>    public Set<String> scan(String matchKey) {        Set<String> keys = redisTemplate.execute((RedisCallback<Set<String>>) connection -> {            Set<String> keysTmp = new HashSet<>();            Cursor<byte[]> cursor = connection.scan(new ScanOptions.ScanOptionsBuilder().match("*" + matchKey + "*").count(1000).build());            while (cursor.hasNext()) {                keysTmp.add(new String(cursor.next()));            }            return keysTmp;        });         return keys;    }我的代码成品, 对count的理解还是不深,待挖掘:    public Set<String> scanKeys(String keyPattern) {        Set<String> keys = (Set<String>) redisTemplate.execute((RedisCallback<Set<String>>) connection -> {            Set<String> keysTmp = new HashSet<>();            ScanOptions scanOptions = new ScanOptions.ScanOptionsBuilder().match(keyPattern).build();            Cursor<byte[]> cursor = connection.scan(scanOptions);            while (cursor.hasNext()) {                keysTmp.add(new String(cursor.next()));            }            return keysTmp;        });        return keys;    }>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>


主要还是安装es7的问题.https://blog.csdn.net/u011265001/article/details/100084335?depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1&utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-1
NG开启basic 认证官方的文档https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/老外写的, 不错~~~https://www.tecmint.com/setup-nginx-basic-http-authentication/
yum install nginx, --->安装在了 /usr/sbin/nginx, 配置文件在/etc/nginx/中
htpasswd /etc/nginx/.htpasswd kibanatips: 这个工具的安装# yum install httpd-tools[RHEL/CentOS]$ sudo apt install apache2-utils[Debian/Ubuntu]

然后交互式,输入密码, 就生成了密码文件$ cat /xxxx/.htpasswduser1:$apr1$/woC1jnP$KAh0SsVn5qeSMjTtn0E9Q0
配置在nginx的配置文件中location /api {    auth_basic           “Administrator’s Area”;    auth_basic_user_file /etc/apache2/.htpasswd; }就OK了.https://www.cnblogs.com/silent2012/p/8377837.html 可以参考

官网reverse proxy的配置在这里samplehttps://nginx.org/en/docs/http/ngx_http_proxy_module.html?_ga=2.27153012.815603832.1587469798-958214196.1587469798#example

############################################################################
docker问题解决, 同下:https://www.cnblogs.com/elson-zeng/p/12553329.htmlCentOS7 firewalld docker 端口映射问题，firewall开放端口后，还是不能访问，解决方案# 宿主机ip: 192.168.91.19docker run -itd --name tomcat -p 8080:8080 tomcat /usr/local/apache-tomcat-9.0.30/bin/startup.sh
# 防火墙放开8080端口firewall-cmd --add-port=8080/tcp --permanent
# 问题：发现访问：192.168.91.19:8080 访问不通，关闭firewall后，又可以访问通了# 解决方案，把docker0网卡添加到trusted域firewall-cmd --permanent --zone=trusted --change-interface=br-d2aa50162455# 重启加载配置firewall-cmd --reload# firewall-cmd相关命令：https://www.cnblogs.com/Raodi/p/11625487.html
我的问题是, 命名防火墙禁用掉了9200端口, 别的机器还是可以访问到.解决方法是,https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file#The default location of the configuration file on Linux is /etc/docker/daemon.json. #The --config-file flag can be used to specify a non-default location.docker的daemon.json里配置:"iptables": false   --->把iptable禁用掉,--->直接导致,kibana无法访问elasticsearch了然后再按上面博客, 把docker的一个区域加入了trusted里. --->kibana可以访问elasticsearch了.>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>看来碰到问题的人好多#跟我差不多https://www.jianshu.com/p/69d3ab177655https://www.jianshu.com/p/10c467600ef9


#docker service的配置文件在/usr/lib/systemd/system/docker.service#当时配置docker服务的代理 在这里 /etc/systemd/system/docker.service.d/http-proxy.conf
这里涉及到了3个配置文件:/etc/docker/daemon.json                 #守护进程各种配置, 配置项非常多, 包括docker镜像站点等/usr/lib/systemd/system/docker.service  #服务配置文件/etc/systemd/system/docker.service.d/http-proxy.conf   #配置代理

所以, 需要学习很多相关的知识, 算是还债. 因为一直没有搞得非常通透, 导致解决问题磕磕绊绊1.systemctl命令相关, 启动脚本在哪里? 配置文件在哪里2.systemctl daemon相关, 配置文件在哪里, 怎么配的
https://www.cnblogs.com/brucewhite/p/11608853.htmlsystemctl --help  查看帮助!!!!!!!systemctl  enable  |  disable |  is-enabled  | status  | is-active  unitsystemctl   get-default|set-default graphical.target|multi-user.target islate 在线切换模式  systemctl reload-daemon  加载新的unit 配置文件  systemd的unit 的配置文件位置:redhat系列： 目录:/usr/lib/systemd/system/ 下 enable 是在 /etc/systemd/system/multi-user.target.wants/  这个目录下 做 unit 配置文件的软链：
3.linux自动启动的几种方式, init.d, chkconfig, service等等几个概念https://www.cnblogs.com/liuxia912/p/10960610.html#CentOS 7的服务systemctl脚本存放在：/usr/lib/systemd/，有系统（system）和用户（user）之分，#像需要开机不登陆就能运行的程序，还是存在系统服务里吧，即：/usr/lib/systemd/system目录下#对于那些支持 Systemd 的软件，安装的时候，会自动在/usr/lib/systemd/system目录添加一个配置文件。
在配置vnc服务器的时候也碰到了这个systemctl daemon-reload
>>待续
4.防火墙的几种配置, firewalld和iptables 
>>>>>>>>>>>>>>>>>>>>>>>>>>>$ firewall-cmd --zone=public --remove-port=10050/tcp##这一句 runtime-to_permanent$ firewall-cmd --runtime-to-permanent$ firewall-cmd --reload >>>>>>>>>>>>>>>>>>>>>>>>>>>

https://www.tecmint.com/start-stop-disable-enable-firewalld-iptables-firewall/下面几个课题有空再看Understanding IPtables Firewall Basics and TipsConfigure Iptables Firewall in LinuxConfigure FirewallD in LinuxUseful FirewallD Rules to Manage Firewall in LinuxHow to Control Network Traffic Using FirewallD and Iptables
# Also, you can mask the firewall service # which creates a symbolic link of the firewall.service to /dev/null, # thus disabling the service.# This is reverse of masking the service. # This removes the symlink of the service created during masking, # thus re-enabling the service.systemctl statrt/stop/restart/disable/enable/mask/unmask/status firewalldfirewall-cmd --state

systemctl restart firewalld.service(为什么要加一个.service, 是什么意思?)systemctl is-enabled firewalld.service          #查看服务是否开机启动systemctl list-unit-files|grep enabled          #查看已启动的服务列表systemctl --failed                              #查看启动失败的服务列表  
关于iptables#How to Start/Stop and Enable/Disable IPtables Servicesystemctl start/stop/restart/disable/enable/status iptablesservice iptables start/stop/save/status
#ubuntu/debian?sudo ufw enable/disable/status
iptables -L -n -v
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>关于firewalld, 这个老外写得太好了!https://www.liquidweb.com/kb/an-introduction-to-firewalld/
firewall-cmd --permanent --add-port=22/TCPfirewall-cmd --permanent --add-port=53/UDPfirewall-cmd --permanent --remove-port=444/tcpfirewall-cmd --permanent --add-service=sshfirewall-cmd --permanent --add-service=httpfirewall-cmd --permanent --remove-service=mysql
#whitelist an ip addressfirewall-cmd --permanent --add-source=192.168.1.100#以CIDR地址形式加一个范围, 这是最近学到的.firewall-cmd --permanent --add-source=192.168.1.0/24firewall-cmd --permanent --remove-source=192.168.1.100
Block an IP Address#As the firewall-cmd tool is mostly used for opening or allowing access, #rich rules are needed to block an IP. #Rich rules are similar in form to the way iptables rules are written.firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.100' reject"#CIDR rangefirewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.0/24' reject"
Whitelist an IP Address for a Specific Port (More Rich Rules)firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'firewall-cmd --permanent --remove-rich-rule='rule family="ipv4" source address="192.168.1.100" port protocol="tcp" port="3306" accept'
#savefirewall-cmd --reloadfirewall-cmd --runtime-to-permanent#showfirewall-cmd --list-all
Bonus #1 :: Firewalld GUI Configuration You can find these under Applications > Sundry menu on CentOS 7.x and RedHat 7.x servers that have graphical access to. To install this application, from the command line, you can run the command:sudo yum install firewall-config
Bonus #2 :: Firewalld Direct Rules/etc/firewalld/direct.xml
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>man 1 systemctlSYSTEMCTL(1)                  systemctl                 SYSTEMCTL(1)
NAME       systemctl - Control the systemd system and service manager
SYNOPSIS       systemctl [OPTIONS...] COMMAND [NAME...]
DESCRIPTION       systemctl may be used to introspect and control the state of       the "systemd" system and service manager. Please refer to       systemd(1) for an introduction into the basic concepts and       functionality this tool manages.
>>>加固Linuxhttps://www.liquidweb.com/kb/security-for-your-linux-server/
>>>>>中文的在这里, 比较冗长, 没有好好排版, 内容不错.https://www.linuxidc.com/Linux/2018-01/150072.htmfirewall daemon 独立于 system-config-firewall，但二者不能同时使用。如果你想使用自己的 iptables 和 ip6tables 静态防火墙规则, 那么请安装 iptables-services 并且禁用 firewalld ，启用 iptables 和ip6tables:yum install iptables-servicessystemctl mask firewalld.servicesystemctl enable iptables.servicesystemctl enable ip6tables.service
静态防火墙规则配置文件是 /etc/sysconfig/iptables 以及 /etc/sysconfig/ip6tables .
什么是区域？网络区域定义了网络连接的可信等级。这是一个一对多的关系，这意味着一次连接可以仅仅是一个区域的一部分，而一个区域可以用于很多连接。
>>>https://www.cnblogs.com/Raodi/p/11625487.html#_label1
>>>>>>>>>>>>>>>>>>>>>>>>若 Nginx 收到请求为 https://ngxin_server_name/hello/world而 Nginx 代理的路径为 /hello/（即在 location /hello/ 内设置代理）则不以 / 结尾的被代理服务器收到的请求路径是 /hello/world以 / 结尾的被代理服务器收到的请求路径是 /world如果是为了在同一个域名下以不同路径分配不同的APP应选择后者以 / 结尾原文链接：https://blog.csdn.net/randomparty/java/article/details/80961189