Kubernetes 外部访问集群内部服务

一、部署一个内部dns

kubernetes内部是有dns的，可以解析集群内部service，应用之间可以通过service名称连接调用。但是节点本身不能直接解析service名称，只能联通service或者pod的ip。

# kubectl get svc -n kube-system
NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                        AGE
kube-dns             ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP         30d
​
​
# nslookup kube-dns.kube-system.svc.cluster.local
Server:     114.114.114.114
Address:    114.114.114.114#53
​
** server can't find kube-dns.kube-system.svc.cluster.local: NXDOMAIN

节点是可以直接和集群中的ip互通的

# ping 10.96.0.10
PING 10.96.0.10 (10.96.0.10) 56(84) bytes of data.
64 bytes from 10.96.0.10: icmp_seq=1 ttl=64 time=0.188 ms
64 bytes from 10.96.0.10: icmp_seq=2 ttl=64 time=0.081 ms
​
# telnet 10.96.0.10 53
Trying 10.96.0.10...
Connected to 10.96.0.10.
Escape character is '^]'.

可以在kubernetes集群内部再部署一个DNS，将集群内部是service名称的解析指向集群中coreDNS。

# dig @10.96.0.10 -p 53 kube-dns.kube-system.svc.cluster.local
​
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el8 <<>> @10.96.0.10 -p 53 kube-dns.kube-system.svc.cluster.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24191
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
​
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: af23a208ee4cfce3 (echoed)
;; QUESTION SECTION:
;kube-dns.kube-system.svc.cluster.local.    IN A
​
;; ANSWER SECTION:
kube-dns.kube-system.svc.cluster.local. 5 IN A  10.96.0.10
​
;; Query time: 3 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Thu Nov 12 15:47:35 CST 2020
;; MSG SIZE  rcvd: 133​

10.96.0.10：集群内部coreDNS的servcie_ip

下载一个coreDNS

# wget https://github.com/coredns/coredns/releases/download/v1.8.0/coredns_1.8.0_linux_amd64.tgz
​
# tar -zxvf coredns_1.8.0_linux_amd64.tgz

创建一个Corefile

cluster.local {
    forward . 10.96.0.10
    log
}
​
.:53 {
    forward . 114.114.114.114
    log
    errors
    cache
}

当请求cluster.local这个域名的解析请求时，coredns就会转发到kubernetes集群内部的10.96.0.10
其他请求还是走114.114.114.114

coredns.service

/usr/lib/systemd/system/coredns.service

[Unit]
Description=CoreDNS DNS server
Documentation=https://coredns.io
After=network.target
  
[Service]
PermissionsStartOnly=true
LimitNOFILE=1048576
LimitNPROC=512
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
User=coredns
WorkingDirectory=~
ExecStart=/usr/local/bin/coredns -conf /data/coredns/Corefile
ExecReload=/bin/kill -SIGUSR1 $MAINPID
Restart=on-failure
  
[Install]
WantedBy=multi-user.target

groupadd coredns
useradd -g coredns coredns<br>
systemctl daemon-reload
systemctl start coredns
systemctl enable coredns

验证

在集群集群节点中测试是否能解析集群中的service名称

# dig @192.168.10.243 -p 53 kube-dns.kube-system.svc.cluster.local
​
; <<>> DiG 9.11.13-RedHat-9.11.13-6.el8_2.1 <<>> @192.168.10.243 -p 53 kube-dns.kube-system.svc.cluster.local
; (1 server found)
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56392
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
​
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 555ea5fc88fd299c (echoed)
;; QUESTION SECTION:
;kube-dns.kube-system.svc.cluster.local.    IN A
​
;; ANSWER SECTION:
kube-dns.kube-system.svc.cluster.local. 5 IN A  10.96.0.10
​
;; Query time: 131 msec
;; SERVER: 192.168.10.243#53(192.168.10.243)
;; WHEN: Thu Nov 12 16:11:14 CST 2020
;; MSG SIZE  rcvd: 133

可以看到节点能直接解析集群内部的service名称了，修改节点服务器的dns，就能直接联通service

# ping kube-dns.kube-system.svc.cluster.local
PING kube-dns.kube-system.svc.cluster.local (10.96.0.10) 56(84) bytes of data.
64 bytes from node01 (10.96.0.10): icmp_seq=1 ttl=64 time=0.106 ms
64 bytes from node01 (10.96.0.10): icmp_seq=2 ttl=64 time=0.062 ms
64 bytes from node01 (10.96.0.10): icmp_seq=3 ttl=64 time=0.064 ms
64 bytes from node01 (10.96.0.10): icmp_seq=4 ttl=64 time=0.087 ms

二、集群外部调整

调整外部机器的dns，将dns地址改为刚才部署的coredns的ip，这样本机就能解析集群内部service名称（windows）

C:\Windows\system32>nslookup kube-dns.kube-system.svc.cluster.local
服务器:  UnKnown
Address:  192.168.10.243
​
名称:    kube-dns.kube-system.svc.cluster.local
Address:  10.96.0.10

此时还是ping不通service，因为集群外部没有到service的路由，这需要添加到service的路由

查看kubernetes集群的service网段

/usr/lib/systemd/system/kube-apiserver.service
 
service-cluster-ip-range=10.96.0.0/12
添加路由
 
route ADD 10.96.0.0/12 192.168.10.243 -p
​
查看
route PRINT

-p：参数，永久静态路由

测试互通

C:\Windows\system32>ping kube-dns.kube-system.svc.cluster.local
​
正在 Ping kube-dns.kube-system.svc.cluster.local [10.96.0.10] 具有 32 字节的数据:
来自 10.96.0.10 的回复: 字节=32 时间<1ms TTL=64
来自 10.96.0.10 的回复: 字节=32 时间<1ms TTL=64
来自 10.96.0.10 的回复: 字节=32 时间<1ms TTL=64