k8s 1.28.2 集群部署 docker registry 接入 MinIO 存储
目录
docker registry 部署
生成 htpasswd 文件
<username> <password>改成自己想配置的，如果密码有特殊字符，要用单引号包起来
docker run --rm \
docker.m.daocloud.io/httpd:latest \
htpasswd -Bbn <username> <password> > htpasswd
生成 secret 文件
kubectl create secret generic docker-registry-auth \
-n registry \
--from-file=htpasswd
生成 registry 配置文件
因为涉及到 MinIO 的
accesskey和secretkey，这里采用 secret 的方式来生成配置文件
---
apiVersion: v1
kind: Secret
metadata:
name: docker-registry-cm
namespace: registry
stringData:
config.yml: |-
version: 0.1
log:
level: info
fields:
service: registry
storage:
delete:
enabled: true
cache:
blobdescriptor: inmemory
s3:
accesskey: wJpkHB8rznvZBRLfKmBz
secretkey: ZHIyklv5tktYvGR0iFqBiL9NKh7JKbhyDR9SNAYp
region: default
regionendpoint: http://minio.api.devops.icu
forcepathstyle: true
accelerate: false
bucket: docker-registry
encrypt: false
secure: false
v4auth: true
chunksize: 5242880
multipartcopymaxconcurrency: 10
http:
addr: :5000
debug:
addr: :5001
prometheus:
enabled: true
path: /metrics
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
auth:
htpasswd:
realm: basic-realm
path: /auth/htpasswd
type: Opaque
创建 service
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: docker-registry
name: docker-registry-svc
namespace: registry
spec:
ports:
- name: http
port: 5000
targetPort: http
- name: http-metrics
port: 5001
targetPort: http-metrics
selector:
app.kubernetes.io/name: docker-registry
type: ClusterIP
创建 statefulset
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
labels:
app.kubernetes.io/name: docker-registry
name: docker-registry
namespace: registry
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: docker-registry
serviceName: docker-registry-svc
template:
metadata:
labels:
app.kubernetes.io/name: docker-registry
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchLabels:
app.kubernetes.io/name: docker-registry
topologyKey: kubernetes.io/hostname
weight: 1
containers:
- image: docker.m.daocloud.io/registry:2.8.3
livenessProbe:
failureThreshold: 60
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: http
timeoutSeconds: 1
name: docker-registry
ports:
- containerPort: 5000
name: http
- containerPort: 5001
name: http-metrics
readinessProbe:
failureThreshold: 60
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: http
timeoutSeconds: 1
resources:
limits:
cpu: 2000m
memory: 2.5Gi
requests:
cpu: 100m
memory: 100Mi
startupProbe:
failureThreshold: 60
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
tcpSocket:
port: http
timeoutSeconds: 1
volumeMounts:
- mountPath: /etc/docker/registry
name: config
- mountPath: /auth
name: auth
terminationGracePeriodSeconds: 30
volumes:
- name: config
secret:
secretName: docker-registry-cm
- name: auth
secret:
secretName: docker-registry-auth
创建 ingress
没有 ingress 可以开 nodeport 来实现
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: 5G
name: docker-registry
namespace: registry
spec:
ingressClassName: nginx
rules:
- host: registry.devops.icu
http:
paths:
- backend:
service:
name: docker-registry-svc
port:
number: 5000
path: /
pathType: Prefix
验证 docker registry
/etc/docker/daemon.json 增加 registry 地址
"insecure-registries": ["ip:端口"]
# 或者
"insecure-registries": ["域名"]
登录 docker registry
docker login http://registry.devops.icu
修改 tag
docker tag docker.m.daocloud.io/registry:2.8.3 registry.devops.icu/registry:2.8.3
上传镜像
docker push registry.devops.icu/registry:2.8.3
docker registry 监控
grafana id：9621
prometheus 配置文件参考
- job_name: docker-registry
kubernetes_sd_configs:
- role: endpoints
relabel_configs:
- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name]
regex: registry;docker-registry-svc
action: keep
- source_labels: [__meta_kubernetes_pod_ip]
regex: (.+)
target_label: __address__
replacement: ${1}:5001
- source_labels: [__meta_kubernetes_endpoints_name]
action: replace
target_label: endpoint
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: pod
- source_labels: [__meta_kubernetes_service_name]
action: replace
target_label: service
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: namespace
docker registry ui
Github 项目地址：Joxit/docker-registry-ui-2.5.7
相关的变量和参数详见：available-options
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: docker-registry-ui
name: docker-registry-ui-svc
namespace: registry
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
app.kubernetes.io/name: docker-registry-ui
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: docker-registry-ui
name: docker-registry-ui
namespace: registry
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: docker-registry-ui
template:
metadata:
labels:
app.kubernetes.io/name: docker-registry-ui
spec:
containers:
- env:
- name: SINGLE_REGISTRY
value: "true"
- name: SHOW_CATALOG_NB_TAGS
value: "true"
- name: REGISTRY_SECURED
value: "true"
- name: NGINX_PROXY_PASS_URL
value: http://docker-registry-svc.registry.svc.cluster.local:5000
- name: NGINX_PROXY_HEADER_Authorization
value: $http_authorization
image: joxit/docker-registry-ui:2.5.7
imagePullPolicy: IfNotPresent
name: docker-registry-ui
securityContext:
fsGroup: 101
runAsUser: 101
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: docker-registry-ui
namespace: registry
spec:
ingressClassName: nginx
rules:
- host: registry.ui.devops.icu
http:
paths:
- backend:
service:
name: docker-registry-ui-svc
port:
number: 8080
path: /
pathType: Prefix
