十一章 Kubernetes的服务发现插件--coredns
1、前言
- 简单来说，服务发现就是服务（应用）之间相互定位的过程；
- 服务发现并非云计算时代独有的，传统的单体架构时代也会用到，以下应用场景更加需要服务发现：
服务（应用）的动态性强；
服务（应用）更新发布频繁；
服务（支持自动）。 - 在k8s集群中pod的IP是不断变化的，所以如何解决这种东问题呢：
抽象出了service资源，通过标签选择器，关联一组pod；
抽象出了集群网络，通过相对固定的“集群（service/Cluster）IP”是服务介入固定IP； - 那么如何自动关联service资源的“名称”和“集群网络IP”，从而达到服务被集群自动发现的目的呢：
考虑传统DNS模型：hdss7-21.host.com -->10.4.7.21；
能否在k8s中建立这样的模型：nginx-ds-->192.168.0.5。
2、部署k8s的内网资源配置清单http服务
2.1 更新域名解析
在10.4.7.11域名服务器上配置新域名解析，注意要前滚一个序列号
[root@hdss7-11 ~]# vim /var/named/od.com.zone
$ORIGIN od.com.
$TTL 600 ; 10 minutes
@ IN SOA dns.od.com. dnsadmin.od.com. (
2020010503 ; serial
10800 ; refresh (3 hours)
900 ; retry (15 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS dns.od.com.
$TTL 60 ; 1 minute
dns A 10.4.7.11
harbor A 10.4.7.200
k8s-yaml A 10.4.7.200
[root@hdss7-11 ~]# systemctl restart named
[root@hdss7-11 ~]# dig -t A k8s-yaml.od.com @10.4.7.11 +short
10.4.7.200
2.2 添加nginx代理
在运维主机10.4.7.200上部署一个nginx的虚拟主机，用以提供k8s统一的资源配置清单访问入口。
配置nginx
[root@hdss7-200 ~]# vim /etc/nginx/conf.d/k8s-yaml.od.com.conf
server {
listen 80;
server_name k8s-yaml.od.com k8s-yaml.grep.pro;
location / {
autoindex on;
default_type text/plain;
root /data/k8s-yaml;
}
}
[root@hdss7-200 ~]# mkdir -p /data/k8s-yaml
[root@hdss7-200 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@hdss7-200 ~]# nginx -s reload
[root@hdss7-200 ~]# cd /data/k8s-yaml/
[root@hdss7-200 k8s-yaml]# ls
[root@hdss7-200 k8s-yaml]# mkdir coredns
访问浏览器http://k8s-yaml.od.com/
3、部署coredns
2.1、准备镜像
[root@hdss7-200 k8s-yaml]# cd /data/k8s-yaml/coredns/
[root@hdss7-200 coredns]# docker pull coredns/coredns:1.6.1
[root@hdss7-200 coredns]# docker tag c0f6e815079e harbor.od.com/public/coredns:v1.6.1
[root@hdss7-200 coredns]# docker push harbor.od.com/public/coredns:v1.6.1
或者coredns]# docker push !$
2.2、准备资源配置清单
cm.yaml中 forward . 10.4.7.11为 集群的dns地址也是coredns的上游dns地址
svc.yaml中 clusterIP: 192.168.0.2地址为集群中kubelet定义好的集群dns地址
在10.4.7.200上部署
[root@hdss7-200 coredns]# cd /data/k8s-yaml/coredns/
配置rbac.yaml
[root@hdss7-200 coredns]# vim rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: coredns
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
kubernetes.io/bootstrapping: rbac-defaults
addonmanager.kubernetes.io/mode: Reconcile
name: system:coredns
rules:
- apiGroups:
- ""
resources:
- endpoints
- services
- pods
- namespaces
verbs:
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
addonmanager.kubernetes.io/mode: EnsureExists
name: system:coredns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:coredns
subjects:
- kind: ServiceAccount
name: coredns
namespace: kube-system
配置cm.yaml
[root@hdss7-200 coredns]# vim cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
log
health
ready
kubernetes cluster.local 192.168.0.0/16
forward . 10.4.7.11
cache 30
loop
reload
loadbalance
}
配置dp.yaml
[root@hdss7-200 coredns]# vim dp.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: coredns
kubernetes.io/name: "CoreDNS"
spec:
replicas: 1
selector:
matchLabels:
k8s-app: coredns
template:
metadata:
labels:
k8s-app: coredns
spec:
priorityClassName: system-cluster-critical
serviceAccountName: coredns
containers:
- name: coredns
image: harbor.od.com/public/coredns:v1.6.1
args:
- -conf
- /etc/coredns/Corefile
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
dnsPolicy: Default
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
配置svc.yaml
[root@hdss7-200 coredns]# vim svc.yaml
apiVersion: v1
kind: Service
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: coredns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "CoreDNS"
spec:
selector:
k8s-app: coredns
clusterIP: 192.168.0.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
- name: metrics
port: 9153
protocol: TCP
2.3、依次执行创建资源
在其中一个运算节点操作，此处以10.4.7.21为例
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/rbac.yaml
serviceaccount/coredns created
clusterrole.rbac.authorization.k8s.io/system:coredns created
clusterrolebinding.rbac.authorization.k8s.io/system:coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/cm.yaml
configmap/coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/dp.yaml
deployment.apps/coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/svc.yaml
service/coredns created
2.4、在kube-system中查询资源
[root@hdss7-21 ~]# kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/coredns-64f49f5655-xs9dq 1/1 Running 0 114s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/coredns ClusterIP 192.168.0.2 <none> 53/UDP,53/TCP,9153/TCP 108s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 1/1 1 1 114s
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-64f49f5655 1 1 1 114s
2.5、检查
[root@hdss7-21 ~]# dig -t A www.baidu.com @192.168.0.2 +short
www.a.shifen.com.
39.156.66.14
39.156.66.18
[root@hdss7-21 ~]# dig -t A hdss7-200.host.com @192.168.0.2 +short
10.4.7.200
[root@hdss7-21 ~]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 9d <none>
创建一个service
[root@hdss7-21 ~]# kubectl create deployment nginx-dp --image=harbor.od.com/public/nginx:v1.7.9
deployment.apps/nginx-dp created
[root@hdss7-21 ~]# kubectl expose deployment nginx-dp --port=80 --target-port=80
service/nginx-dp exposed
[root@hdss7-21 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-dp-86678bb55c-tklvc 1/1 Running 0 19s 172.7.21.4 hdss7-21.host.com <none> <none>
nginx-ds-2rj9d 1/1 Running 0 84m 172.7.21.2 hdss7-21.host.com <none> <none>
nginx-ds-cnz5d 1/1 Running 0 95m 172.7.22.3 hdss7-22.host.com <none> <none>
[root@hdss7-21 ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 9d
nginx-dp ClusterIP 192.168.191.8 <none> 80/TCP 61s
解析域名
[root@hdss7-21 ~]# dig -t A nginx-dp.default.svc.cluster.local @192.168.0.2 +short
192.168.191.8
在容器集群内部访问域名
[root@hdss7-21 ~]# kubectl exec -it nginx-dp-86678bb55c-tklvc bash
root@nginx-dp-86678bb55c-tklvc:/# ping nginx-dp.default.svc.cluster.local
PING nginx-dp.default.svc.cluster.local (192.168.191.8): 48 data bytes
56 bytes from 192.168.191.8: icmp_seq=0 ttl=64 time=0.281 ms
56 bytes from 192.168.191.8: icmp_seq=1 ttl=64 time=0.081 ms
root@nginx-dp-86678bb55c-tklvc:/# curl nginx-dp.default.svc.cluster.local
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
如下/etc/resolv.conf，可以直接访问短域名
root@nginx-dp-86678bb55c-tklvc:/# cat /etc/resolv.conf
nameserver 192.168.0.2
search default.svc.cluster.local svc.cluster.local cluster.local host.com
options ndots:5
root@nginx-dp-86678bb55c-tklvc:/# ping nginx-dp
PING nginx-dp.default.svc.cluster.local (192.168.176.18): 48 data bytes
56 bytes from 192.168.176.18: icmp_seq=0 ttl=64 time=0.135 ms
56 bytes from 192.168.176.18: icmp_seq=1 ttl=64 time=0.110 ms
但是容器外非集群却无法访问此域名
root@nginx-dp-86678bb55c-tklvc:/# exit
exit
[root@hdss7-21 ~]# ping nginx-dp.default.svc.cluster.local
ping: nginx-dp.default.svc.cluster.local: 未知的名称或服务
[root@hdss7-21 ~]# curl nginx-dp.default.svc.cluster.local
curl: (6) Could not resolve host: nginx-dp.default.svc.cluster.local; 未知的错误
目前coredns已经完成了服务的域名和cluster的IP做了一个自动关联，实现了服务的自动发现，下一节会学习如何在集群外暴露服务域名
2.6 容器可以访问外网
注意：并且之前我们用的192.168.0.2的dns在容器中时无法访问外网的，部署好coredns后也可以对外网进行访问了
root@nginx-dp-86678bb55c-tklvc:/# ping baidu.com
PING baidu.com (39.156.69.79): 48 data bytes
56 bytes from 39.156.69.79: icmp_seq=0 ttl=52 time=10.574 ms
56 bytes from 39.156.69.79: icmp_seq=1 ttl=52 time=9.379 ms
^C--- baidu.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 9.379/9.976/10.574/0.598 ms
