minikube上部署jenkins
minikube上部署jenkins
将jenkins部署到minikube上，并提供外网访问
相关minikube安装和操作参考：
- https://blog.iwiki.ink/archives/ubuntu安装minikubekubernetes
- https://blog.iwiki.ink/archives/minikube-qi-dong-dashboard-bing-ke-gong-wai-wang-fang-wen
jenkins部署到minikube参考官方文档（跟官方走总不会翻车🤣）：
1.安装jenkins
官方已经提供了jenkins部署所需的主要yaml文件，在：https://github.com/scriptcamp/kubernetes-jenkins
这里主要讲解下每个文件的作用
1.1 创建namespace
kubectl create namespace devops-tools
此步非必需，为了方便管理，用namespace区分开
1.2 配置集群权限 serviceAccount.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: jenkins-admin
rules:
- apiGroups: [""]
resources: ["*"]
verbs: ["*"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins-admin
namespace: devops-tools
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: jenkins-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins-admin
subjects:
- kind: ServiceAccount
name: jenkins-admin
namespace: devops-tools
这个文件主要是服务于集群权限控制，这个直接照官方来好了。注意namespace一致即可
# 执行如下命令生效
kubectl apply -f serviceAccount.yaml
1.3 配置挂载数据卷 volume.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-pv-volume
labels:
type: local
spec:
storageClassName: local-storage
claimRef:
name: jenkins-pv-claim
namespace: devops-tools
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
local:
path: /mnt
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- worker-node01
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-pv-claim
namespace: devops-tools
spec:
storageClassName: local-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
注意下node，worker-node01这个值需要跟着实际的node走，minikube部署那这个值就是minikube，k8s的话，需要执行
kubectl get nodes
才能知道node信息
这里的path: /mnt一定要重点注意，它挂载是挂载到minikube虚拟机环境中的地址，我这里是用的docker
myPc@myPc:/mnt$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f066afd33376 registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.36 "/usr/local/bin/entr…" 27 hours ago Up 21 minutes 127.0.0.1:49157->22/tcp, 127.0.0.1:49156->2376/tcp, 127.0.0.1:49155->5000/tcp, 127.0.0.1:49154->8443/tcp, 127.0.0.1:49153->32443/tcp minikube
myPc@myPc:/mnt$ docker exec -it minikube /bin/bash
root@minikube:/# ls
Release.key bin boot data dev docker.key etc home kic.txt kind lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var
root@minikube:/# cd mnt/
root@minikube:/mnt# ls
config.xml hudson.model.UpdateCenter.xml jobs nodes queue.xml.bak secret.key.not-so-secret updates users
copy_reference_file.log jenkins.telemetry.Correlator.xml nodeMonitors.xml plugins secret.key secrets userContent war
看看上面的输出，你应该懂了吧😂
你可能在想，要怎么才能把数据挂载到宿主机上呢？可以使用minikube mount命令
# 例如
minkube mount /mnt:/mnt
但是！但是！到生产集群环境下，每台机器都挂载到自己的宿主机上，想想也太扯了，所以要统一数据文件持久化到一台固定的机器上，方便维护。这里提示下可以考虑NFS（网络文件系统）技术
执行如下命令生效
kubectl create -f volume.yaml
1.4 部署 deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
namespace: devops-tools
spec:
replicas: 1
selector:
matchLabels:
app: jenkins-server
template:
metadata:
labels:
app: jenkins-server
spec:
securityContext:
fsGroup: 1000
runAsUser: 1000
serviceAccountName: jenkins-admin
containers:
- name: jenkins
image: jenkins/jenkins:lts
resources:
limits:
memory: "2Gi"
cpu: "1000m"
requests:
memory: "500Mi"
cpu: "500m"
ports:
- name: httpport
containerPort: 8080
- name: jnlpport
containerPort: 50000
livenessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 90
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
volumeMounts:
- name: jenkins-data
mountPath: /var/jenkins_home
volumes:
- name: jenkins-data
persistentVolumeClaim:
claimName: jenkins-pv-claim
这里的volumeMounts就是关联到volume.yaml的配置,意思是讲jenkins容器中的 /var/jenkins_home地址挂载到volume.yaml中配置的/mnt地址，类似于docker的-v数据参数，只不过这里挂载也只是挂载到minikube虚拟环境中
执行如下命令生效
kubectl apply -f deployment.yaml
1.5 对外暴露服务 service.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
namespace: devops-tools
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8080'
spec:
selector:
app: jenkins-server
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 32000
执行如下命令生效
kubectl apply -f service.yaml
该文件是以nodePort的形式向外暴露服务的，效果等于
kubectl expose deployment jenkins --port=32000 --type=NodePort
执行完成后你就可以以任一node的ip地址+32000端口去访问jenkins了，这个32000端口可自行修改
如果不知道端口，可以执行
minikube service jenkins-service -n=devops-tools
查看URL信息，这个命令也会自动打开浏览器，ssh下会报错
2.配置外网可达
执行完上面所有的步骤后外网依旧没法访问，只能宿主机上访问。官方也提供了相应的解决方案，不过我这里是用的是nginx反向代理，安装nginx
sudo apt-get install nginx
然后
sudo vim /etc/nginx/conf.d/jenkins.conf
jenkins.conf内容如下
upstream jenkins {
server 192.168.49.2:32000;
}
server {
listen 8080;
server_name 192.168.43.18;
#charset koi8-r;
#access_log /var/log/nginx/log/showdoc.access.log main;
location / {
proxy_pass http://jenkins;
add_header Access-Control-Allow-Origin *; #允许跨域
add_header Access-Control-Allow-Headers X-Requested-With;
add_header Access-Control-Allow-Methods GET,POST,OPTIONS;
}
}
upstream中的ip和端口为NodePort的地址
server中配置宿主机的ip和端口，配置完后执行
sudo service nginx restart
重启nginx即可，然后就能外网访问到jenkins了
补充
任何时候踩坑都没有一蹴而就的情况🐶
附上删除命令，等执行错了的时候可以回滚
关于数据卷的
# 用于删除volume.yaml中配置的StorageClass
kubectl delete sc local-storage -n=devops-tools
# 用于删除volume.yaml中配置的PersistentVolume
kubectl delete pv jenkins-pv-volume -n=devops-tools
# 用于删除volume.yaml中配置的PersistentVolume
kubectl delete pvc jenkins-pv-claim -n=devops-tools
关于service的
# 用于删除service ，svc等价于service，缩写
kubectl delete svc jenkins-service -n=devops-tools
关于deployment的
# 用于删除service ，svc等价于service，缩写
kubectl delete deployment jenkins -n=devops-tools
具体的删除可以参考这篇文章：https://www.modb.pro/db/381010
