Kuberntes部署MetalLB负载均衡器
转载自：https://www.oiox.cn/index.php/archives/416/
MetalLB简介
MetalLB是一个为基础 Kubernetes集群提供负载均衡实现的工具，使用标准路由协议。
Kubernetes在基础集群中不提供网络负载均衡器（类型为LoadBalancer的服务）的实现。Kubernetes提供的网络负载平衡器实现都是调用各种IaaS平台（如GCP、AWS、Azure等）的接口代码。如果您没有运行在受支持的IaaS平台上（如GCP、AWS、Azure等），则创建时LoadBalancers将无限期处于“挂起”状态。
在基础集群中，操作员只有两个接口来将用户流量引入他们的集群，“NodePort”和“externalIPs”服务。
这两个选项在生产使用中都有显著的缺点，这使得基础集群成为 Kubernetes 生态系统中的二等公民。
MetalLB旨在通过提供与标准网络设备集成的网络负载均衡器实现来解决这种不平衡，以便基础群集上的external services尽可能“正常工作”。
root@master:~# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master01 Ready control-plane 18h v1.30.2 192.168.1.31 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-master02 Ready control-plane 18h v1.30.2 192.168.1.32 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-master03 Ready control-plane 18h v1.30.2 192.168.1.33 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-node01 Ready <none> 18h v1.30.2 192.168.1.34 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
k8s-node02 Ready <none> 18h v1.30.2 192.168.1.35 <none> Ubuntu 24.04 LTS 6.8.0-35-generic containerd://1.7.18
修改kube-system
如果您正在使用IPVS模式下的kube-proxy，则自Kubernetes v1.14.2起，您必须启用严格的ARP模式。
请注意，如果您使用kube-router作为服务代理，则不需要此操作，因为它默认启用了 strict ARP。
您可以通过编辑当前集群中的kube-proxy配置来实现此操作：
# kubeadm 部署方式 修改kube-system
kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl apply -f - -n kube-system
# 二进制 部署方式 修改kube-system
cat > /etc/kubernetes/kube-proxy.yaml << EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
acceptContentTypes: ""
burst: 10
contentType: application/vnd.kubernetes.protobuf
kubeconfig: /etc/kubernetes/kube-proxy.kubeconfig
qps: 5
clusterCIDR: 172.16.0.0/12,fc00:2222::/112
configSyncPeriod: 15m0s
conntrack:
max: null
maxPerCore: 32768
min: 131072
tcpCloseWaitTimeout: 1h0m0s
tcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: ""
iptables:
masqueradeAll: false
masqueradeBit: 14
minSyncPeriod: 0s
syncPeriod: 30s
ipvs:
strictARP: true
masqueradeAll: true
minSyncPeriod: 5s
scheduler: "rr"
syncPeriod: 30s
kind: KubeProxyConfiguration
metricsBindAddress: 127.0.0.1:10249
mode: "ipvs"
nodePortAddresses: null
oomScoreAdj: -999
portRange: ""
udpIdleTimeout: 250ms
EOF
systemctl restart kube-proxy
systemctl status kube-proxy
部署metallb
# 下载应用包
wget https://mirrors.chenby.cn/https://github.com/metallb/metallb/archive/refs/tags/v0.14.5.tar.gz
tar -zxvf v0.14.5.tar.gz
cd metallb-0.14.5/config/manifests
# 修改镜像地址
sed -i "s#quay.io#quay.chenby.cn#g" metallb-native.yaml
cat metallb-native.yaml | grep image
image: quay.chenby.cn/metallb/controller:v0.14.5
image: quay.chenby.cn/metallb/speaker:v0.14.5
# 执行部署
kubectl apply -f metallb-native.yaml
root@k8s-master01:~# kubectl -n metallb-system get all
NAME READY STATUS RESTARTS AGE
pod/controller-6975f6bf7b-nm2d6 1/1 Running 0 23m
pod/speaker-4jtb4 1/1 Running 0 23m
pod/speaker-fpd6q 1/1 Running 0 23m
pod/speaker-mmfxq 1/1 Running 0 23m
pod/speaker-rxs2b 1/1 Running 0 23m
pod/speaker-sfxvb 1/1 Running 0 23m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/metallb-webhook-service ClusterIP 10.96.95.84 <none> 443/TCP 23m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/speaker 5 5 5 5 5 kubernetes.io/os=linux 23m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/controller 1/1 1 1 23m
NAME DESIRED CURRENT READY AGE
replicaset.apps/controller-6975f6bf7b 1 1 1 23m
root@k8s-master01:~#
创建池
# 新版本metallb使用了CR（Custom Resources），这里我们通过IPAddressPool的CR，进行地址池的定义。
# 如果实例中不设置IPAddressPool选择器L2Advertisement；那么L2Advertisement默认为该实例所有的IPAddressPool相关联。
cat > metallb-config-ipaddresspool.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- 192.168.1.70-192.168.1.79
EOF
# 进行L2关联地址池的绑定。
cat > metallb-config-L2Advertisement.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-system
spec:
ipAddressPools:
- first-pool
EOF
# 执行部署
kubectl apply -f metallb-config-ipaddresspool.yaml
kubectl apply -f metallb-config-L2Advertisement.yaml
测试
# 测试
cat > metallb-nginx.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: default
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
type: LoadBalancer
EOF
# 执行部署
kubectl apply -f metallb-nginx.yaml
查看
# 查看
# EXTERNAL-IP已经获取到IP地址
root@k8s-master01:~# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-648c475cfb-5pvvv 1/1 Running 0 4m24s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx LoadBalancer 10.96.197.147 192.168.1.51 80:30752/TCP 4m24s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 4m24s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-648c475cfb 1 1 1 4m24s
root@k8s-master01:~#
访问
# 访问
root@k8s-master01:~# curl 192.168.1.51
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-master01:~#
