Kubernetes搭建spinnaker服务
背景：
2017-2018年左右的吧，不记得看什么了看到了spinnaker，但是当时真的安装不起来。各种被墙裂。2020年底学习了泽阳大佬的spinnaker实践课程。通过Halyard方式搭建了spinnaker的集群，并与jenkins gitlab harbor k8s完成了集成。2021年初稍微玩了一下，就去整别的事情去了，没有能应用于线上环境。下半年了，jenkins k8s这些的流程现在基本都是清晰了。想把cd从jenkins中剥离出来教给spinnaker了，就重新温习一下spinnaker吧！
关于spinnaker
spinnaker是Netfix公司开源的一款持续部署工具，采用java语言编写，遵循微服务的设计思想，目标是为团队提供灵活的持续部署流水线并提供软件的部署效率
spinnaker的优势
- 支持多云部署
- 自动发布
- 内置部署最佳实践
spinnaker架构
关于spinnaker的架构说明
-
deck-基于浏览器的 UI
-
gate 微服务api网关Spinnaker UI 和所有 api 调用者通过 Gate 与 Spinnaker 通信
-
orca 流水线阶段编排引擎它处理所有临时操作和管道。阅读有关 Orca 服务概述的更多信息
-
clouddriver负责对云提供商的所有变异调用以及索引/缓存所有部署的资源。
-
front50用于持久化应用程序、管道、项目和通知的元数据
-
rosco为各种云提供商生成不可变的 VM 映像（或映像模板）
它用于生成机器映像（例如 GCE 映像 、 AWS AMI 、 Azure VM 映像 ）。它目前包装了 packer ，但将 被扩展以支持用于生成图像的其他机制。
-
igor用于通过 Jenkins 和 Travis CI 等系统中的持续集成作业触发管道，它允许在管道中使用 Jenkins/Travis 阶段
-
echo事件总线它支持发送通知（例如 Slack、电子邮件、SMS），并对来自 Github 等服务的传入 webhook 采取行动。
-
fiat 认证授权中心它用于查询用户对帐户、应用程序和服务帐户的访问权限
-
kayenta 自动金丝雀分析
-
Keel为管理交付提供动力
注：这个还没有用过 -
halyard 配置服务 管理上述每项服务的生命周期。它仅在 Spinnaker 启动、更新和回滚期间与这些服务交互。
服务依赖调用关系：
重要的事情： 这些东西去看官方文档很是详细，比其他的比较详细多了：https://spinnaker.io/docs/reference/architecture/microservices-overview/
Kubernetes搭建spinnaker服务
注：spinnaker的安装方式有helm 和halyard的本地部署方式 这里采用了halyard的方式！。基本过程参照泽阳大佬的spinnaker课程！
本人集群环境为kubernetes1.20.6 rutime使用了containerd并没有采用docker。中间过程尝试了很多次各种失败，先基于docker的方式做一次安装部署。后面剖析一下containerd方式！
基本环境
腾讯云同一vpc内服务器，内网互通，ip为内网地址
| 主机名 | ip | 系统 | 内核 | k8s版本 | |
|---|---|---|---|---|---|
| k8s-master-01 | 10.0.0.41 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-master-02 | 10.0.0.34 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-master-03 | 10.0.0.26 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-01 | 10.0.4.49 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-02 | 10.0.4.48 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-03 | 10.0.4.23 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-04 | 10.0.4.47 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-05 | 10.0.4.32 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | containerd |
| k8s-node-06 | 10.0.4.18 | CentOS Linux 8 | 5.4.134-1.el8.elrepo.x86_64 | v1.21.3 | docker |
| k8s-01 | 10.0.2.17 | CentOS Linux 8 | 4.18.0-305.12.1.el8_4.x86_64 | 不在集群内（但是也是一个测试的k8s集群，故上面的其他pod忽略） | docker(集群外一台运行docker的服务器) |
注：个人尝试containerd运行halyard未能成功，最终使用docker方式运行halyard
基于docker runtime方式部署halyard的方式部署spinnaker
注： 关于halyard的操作都在k8s-01节点操作。另外声明一下k8s-01原主机名为k8s-02使用了hostnamectl set-hostname修改主机名。有些截图或者命令都依然为k8-02，实际为同一个台服务器。xshell早些时候打开10.0.2.17的窗口......
下载镜像，挂载本地配置文件目录，并启动容器
[root@k8s-01 ~]# docker pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
####创建.hall文件夹后面持久化存储spinnaker生成文件
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.hal
###创建.kube文件夹并将集群中的config文件上传到此目录
[root@k8s-01 ~]# mkdir -p /home/spinnaker/.kube
[root@k8s-01 ~]# ls /home/spinnaker/.kube
config
####启动halyard容器
[root@k8s-01 ~]# docker run -itd --name halyard -v /home/spinnaker/.hal:/home/spinnaker/.hal -v /home/spinnaker/.kube:/home/spinnaker/.kube registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
特权身份进入容器关闭gcs
## 以root身份进入容器，修改配置文件
[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0#
## 修改spinnaker.config.input.gcs.enabled = false 。
vi /opt/halyard/config/halyard.yml
spinnaker:
artifacts:
debian: https://dl.bintray.com/spinnaker-releases/debians
docker: gcr.io/spinnaker-marketplace
config:
input:
gcs:
enabled: false
writerEnabled: false
bucket: halconfig
重新启动halyard容器
## 需要重启容器（如果此命令未重启，则需要退出容器然后 docker restart halyard）
bash-5.0# hal shutdown
Halyard Daemon Response: Shutting down, bye...
##重启容器
[root@k8s-01 .kube]# docker start halyard
halyard
上传boms文件到服务器
参照https://github.com/zeyangli/spinnaker-cd-install,这里使用的是https://github.com/zeyangli/spinnaker-cd-install/actions/runs/1368350526 1.26.6的制品：
###通过rz命令上传制品库到运行halyard的服务器，并解压压缩包
[root@k8s-01 work]# ls
1.26.6-Install-Scripts.zip
[root@k8s-01 work]# unzip 1.26.6-Install-Scripts.zip
嗯看到了这个.boms的文件夹，将其copy到/home/spinnaker/.hal/目录下！
[root@k8s-01 1.26.6]# ls .boms/
bom clouddriver deck echo fiat front50 gate igor kayenta monitoring-daemon orca rosco
[root@k8s-01 1.26.6]# cp -Ra .boms/ /home/spinnaker/.hal/
[root@k8s-01 1.26.6]# ls /home/spinnaker/.hal/.boms/
bom clouddriver deck echo fiat front50 gate igor kayenta monitoring-daemon orca rosco
关于镜像的下载
镜像下载泽阳大佬的制品库下载中有下载镜像的脚本：
#!/bin/bash
S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="node01.zy.com node02.zy.com"
## 下载镜像
function GetImages(){
echo -e "\033[43;34m =====GetImg===== \033[0m"
IMAGES=$( cat tagfile.txt)
for image in ${IMAGES}
do
for node in ${NODES}
do
echo -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
ssh ${node} "docker pull ${T_REGISTRY}/${image}"
echo -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
ssh ${node} "docker tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
done
done
for node in ${NODES}
do
echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
ssh ${node} "docker images | grep 'spinnaker-marketplace' "
done
}
GetImages
But 我的集群的运行时是containerd。ctr crictl两个命令的区别有必要重新温习一下。crictl也没法修改标签啊？
#!/bin/bash
S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"
## 下载镜像
function GetImages(){
echo -e "\033[43;34m =====GetImg===== \033[0m"
IMAGES=$( cat tagfile.txt)
for image in ${IMAGES}
do
for node in ${NODES}
do
echo -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
ssh -p 36000 ${node} "crictl pull ${T_REGISTRY}/${image}"
echo -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
done
done
for node in ${NODES}
do
echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
ssh -p 36000 ${node} "crictl images ls| grep 'spinnaker-marketplace' "
done
}
GetImages
所以这个方式就行不通了,然后偶然搜到csdn的---安装篇——用halyard安装Spinnaker。通过在.hall目录下default/service-settings/目录创建对应配置文件。并设置artifactId！
至于service-settings目录为什么在default目录下我也不求甚解泽阳大佬的课程中修改redis为外部redis的时候有这个目录
[root@k8s-2 .hal]# mkdir -p /home/spinnaker/.hal/default/service-settings
[root@k8s-2 .hal]# cd /home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# pwd
/home/spinnaker/.hal/default/service-settings
[root@k8s-2 service-settings]# ls
clouddriver.yml deck.yml echo.yml fiat.yml front50.yml gate.yml igor.yml kayenta.yml orca.yml rosco.yml
[root@k8s-2 service-settings]# cat *
artifactId: docker.io/spinnakercd/clouddriver:8.0.4-20210625060028
artifactId: docker.io/spinnakercd/deck:3.7.2-20210614020020
artifactId: docker.io/spinnakercd/echo:2.17.1-20210429125836
artifactId: docker.io/spinnakercd/fiat:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/front50:0.27.1-20210625161956
artifactId: docker.io/spinnakercd/gate:1.22.1-20210603020019
artifactId: docker.io/spinnakercd/igor:1.16.0-20210422230020
artifactId: docker.io/spinnakercd/kayenta:0.21.0-20210322140019
artifactId: docker.io/spinnakercd/orca:2.20.3-20210630022216
artifactId: docker.io/spinnakercd/rosco:0.25.0-20210422230020
就不修改标签直接使用泽阳大佬docker的镜像仓库里面的镜像了免去下载镜像修改标签的步骤
Halyard配置管理
注： halyard的配置都在k8s-01节点执行默认在halyard容器内
设置Spinnaker版本，--version 指定版本
[root@k8s-01 .kube]# docker exec -it -u root halyard bash
bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
Success
- Edit Spinnaker version
Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
"/home/spinnaker/.hal/config": /home/spinnaker/.hal/config
- Failed to update version.
嗯强调一下 .hall目录要有读写权限啊
[root@k8s-01 1.26.6]# chmod 777 -R /home/spinnaker/.hal/
[root@k8s-01 1.26.6]#
继续指定spinnaker版本并生成配置文件
bash-5.0$ hal config version edit --version local:1.26.6
+ Get current deployment
Success
+ Edit Spinnaker version
Success
+ Spinnaker has been configured to update/install version
"local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
bash-5.0$ ls
config default
bash-5.0$ cat config
currentDeployment: default
deploymentConfigurations:
- name: default
version: local:1.26.6
providers:
appengine:
enabled: false
accounts: []
aws:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
defaultKeyPairTemplate: '{{name}}-keypair'
defaultRegions:
- name: us-west-2
defaults:
iamRole: BaseIAMRole
ecs:
enabled: false
accounts: []
azure:
enabled: false
accounts: []
bakeryDefaults:
templateFile: azure-linux.json
baseImages: []
dcos:
enabled: false
accounts: []
clusters: []
dockerRegistry:
enabled: false
accounts: []
google:
enabled: false
accounts: []
bakeryDefaults:
templateFile: gce.json
baseImages: []
zone: us-central1-f
network: default
useInternalIp: false
huaweicloud:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
kubernetes:
enabled: false
accounts: []
tencentcloud:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
oracle:
enabled: false
accounts: []
bakeryDefaults:
templateFile: oci.json
baseImages: []
cloudfoundry:
enabled: false
accounts: []
deploymentEnvironment:
size: SMALL
type: LocalDebian
imageVariant: SLIM
updateVersions: true
consul:
enabled: false
vault:
enabled: false
customSizing: {}
sidecars: {}
initContainers: {}
hostAliases: {}
affinity: {}
tolerations: {}
nodeSelectors: {}
gitConfig:
upstreamUser: spinnaker
livenessProbeConfig:
enabled: false
haServices:
clouddriver:
enabled: false
disableClouddriverRoDeck: false
echo:
enabled: false
persistentStorage:
azs: {}
gcs:
rootFolder: front50
redis: {}
s3:
rootFolder: front50
oracle: {}
features:
auth: false
fiat: false
chaos: false
entityTags: false
metricStores:
datadog:
enabled: false
tags: []
prometheus:
enabled: false
add_source_metalabels: true
stackdriver:
enabled: false
newrelic:
enabled: false
tags: []
period: 30
enabled: false
notifications:
slack:
enabled: false
twilio:
enabled: false
baseUrl: https://api.twilio.com/
github-status:
enabled: false
timezone: America/Los_Angeles
ci:
jenkins:
enabled: false
masters: []
travis:
enabled: false
masters: []
wercker:
enabled: false
masters: []
concourse:
enabled: false
masters: []
gcb:
enabled: false
accounts: []
codebuild:
enabled: false
accounts: []
repository:
artifactory:
enabled: false
searches: []
security:
apiSecurity:
ssl:
enabled: false
uiSecurity:
ssl:
enabled: false
authn:
oauth2:
enabled: false
client: {}
resource: {}
userInfoMapping: {}
saml:
enabled: false
userAttributeMapping: {}
ldap:
enabled: false
x509:
enabled: false
iap:
enabled: false
enabled: false
authz:
groupMembership:
service: EXTERNAL
google:
roleProviderType: GOOGLE
github:
roleProviderType: GITHUB
file:
roleProviderType: FILE
ldap:
roleProviderType: LDAP
enabled: false
artifacts:
bitbucket:
enabled: false
accounts: []
gcs:
enabled: false
accounts: []
oracle:
enabled: false
accounts: []
github:
enabled: false
accounts: []
gitlab:
enabled: false
accounts: []
gitrepo:
enabled: false
accounts: []
http:
enabled: false
accounts: []
helm:
enabled: false
accounts: []
s3:
enabled: false
accounts: []
maven:
enabled: false
accounts: []
templates: []
pubsub:
enabled: false
google:
enabled: false
pubsubType: GOOGLE
subscriptions: []
publishers: []
canary:
enabled: false
serviceIntegrations:
- name: google
enabled: false
accounts: []
gcsEnabled: false
stackdriverEnabled: false
- name: prometheus
enabled: false
accounts: []
- name: datadog
enabled: false
accounts: []
- name: signalfx
enabled: false
accounts: []
- name: aws
enabled: false
accounts: []
s3Enabled: false
- name: newrelic
enabled: false
accounts: []
reduxLoggerEnabled: true
defaultJudge: NetflixACAJudge-v1.0
stagesEnabled: true
templatesEnabled: true
showAllConfigsEnabled: true
spinnaker:
extensibility:
plugins: {}
repositories: {}
webhook:
trust:
enabled: false
stats:
enabled: true
endpoint: https://stats.spinnaker.io
instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
deploymentMethod: {}
connectionTimeoutMillis: 3000
readTimeoutMillis: 5000
bash-5.0$
设置时区
# 设置时区
hal config edit --timezone Asia/Shanghai
S3--no-validate
# 设置存储为s3(后面不用，但是必须配置bug)
hal config storage edit --type s3 --no-validate
访问方式,设置deck与gate的域名
# 访问方式：设置deck与gate的域名
hal config security ui edit --override-base-url http://spinnaker.xxxx.com
hal config security api edit --override-base-url http://spin-gate.xxxx.com
来对比一下执行以上命令后config文件的变化：
做这些对比是为了方便以后自己手动更改配置文件。大佬的可以忽略这些截图步骤。
添加镜像仓库（harbor）和k8s集群账户
开启镜像仓库配置并添加account
bash-5.0$ hal config provider docker-registry enable --no-validate
+ Get current deployment
Success
+ Edit the dockerRegistry provider
Success
+ Successfully enabled dockerRegistry
bash-5.0$ hal config provider docker-registry account add my-harbor-registry \
> --address https://harbor.xxxx.com \
> --username xxxx \
> --password xxxx
+ Get current deployment
Success
+ Add the my-harbor-registry account
Success
Validation in
default.provider.dockerRegistry.my-harbor-registry:
- WARNING Your docker registry has no repositories specified, and
the registry's catalog is empty. Spinnaker will not be able to deploy any images
until some are pushed to this registry.
? Manually specify some repositories for this docker registry to
index.
+ Successfully added account my-harbor-registry for provider
dockerRegistry.
开启kubernetes配置并添加account
bash-5.0$ hal config provider kubernetes enable
+ Get current deployment
Success
+ Edit the kubernetes provider
Success
Validation in default.provider.kubernetes:
- WARNING Provider kubernetes is enabled, but no accounts have been
configured.
+ Successfully enabled kubernetes
bash-5.0$ hal config provider kubernetes account add default \
> --docker-registries my-harbor-registry \
> --context $(kubectl config current-context) \
> --service-account true \
> --omit-namespaces=kube-system,kube-public \
> --provider-version v2 \
> --no-validate
+ Get current deployment
Success
+ Add the default account
Success
+ Successfully added account default for provider kubernetes.
再瞄一眼配置文件config:
指定部署使用account和命名空间，部署方式distributed（分布式）
bash-5.0$ hal config deploy edit \
> --account-name default \
> --type distributed \
> --location spinnaker
看了一眼配置文件应该对应的是deploymentEnvironment下面的配置：
开启一些主要的功能（后期可以再追加）
bash-5.0$ hal config features edit --pipeline-templates true
bash-5.0$ hal config features edit --artifacts true
bash-5.0$ hal config features edit --managed-pipeline-templates-v2-ui true
查看config配置文件对应的为features下开关：
配置与jenkins CI集成
# 配置Jenkins
hal config ci jenkins enable
### JenkinsServer 需要用到账号和密码
hal config ci jenkins master add my-jenkins-master-01 \
--address https://jenkins.xxxx.com \
--username zhangpeng \
--password xxxx
### 启用csrf
hal config ci jenkins master edit my-jenkins-master-01 --csrf true
cat config对应如下：当然了也可以开启travis wercker consourse gcb等ci工具？
配置GitHub/GitLab集成
github的是泽阳大佬的。我这里就只集成了gitlab。github仅供参考在配置文件中也生成一下。方便对比配置文件。token的生成就不用做过多的赘述了！
# GitHub
## 参考：https://spinnaker.io/setup/artifacts/github/
## 创建token https://github.com/settings/tokens
hal config artifact github enable
hal config artifact github account add my-github-account \
--token xxxxxxxxxxxxxxxxxxxxxxx \
--username zeyangli
# GitLab
## https://spinnaker.io/setup/artifacts/gitlab/
## 创建一个个人的token（admin）
hal config artifact gitlab enable
hal config artifact gitlab account add my-gitlab-account \
--token xxxxxxxxxxxxxx
artifacts下找到相关配置
使用外部redis集群
关于redis我是使用的腾讯云的云redis。正常该搞一个密码的。但是没有去仔细看下官方文档，就直接使用了免密的方式！
## service-settings
bash-5.0$ pwd
/home/spinnaker/.hal/default/service-settings
vi .hal/default/service-settings/redis.yml
overrideBaseUrl: redis://10.0.0.31:6379
skipLifeCycleManagement: true
## profiles
## /home/spinnaker/.hal/default/profiless
bash-5.0$ pwd
/home/spinnaker/.hal/default
bash-5.0$ mkdir /home/spinnaker/.hal/default/profiles
bash-5.0$ cd profiles/
bash-5.0$ vi gate-local.yml
redis:
configuration:
secure:
true
使用SQL数据库
mysql我是直接开启了腾讯云的TDSQL-C
Clouddriver服务
创建数据库：
CREATE DATABASE `clouddriver` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT
SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_service'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';
GRANT
SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW
ON `clouddriver`.*
TO 'clouddriver_migrate'@'%' IDENTIFIED BY 'clouddriver@spinnaker.com';
修改配置文件：
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi clouddriver-local.yml
sql:
enabled: true
# read-only boolean toggles `SELECT` or `DELETE` health checks for all pools.
# Especially relevant for clouddriver-ro and clouddriver-ro-deck which can
# target a SQL read replica in their default pools.
read-only: false
taskRepository:
enabled: true
cache:
enabled: true
# These parameters were determined to be optimal via benchmark comparisons
# in the Netflix production environment with Aurora. Setting these too low
# or high may negatively impact performance. These values may be sub-optimal
# in some environments.
readBatchSize: 500
writeBatchSize: 300
scheduler:
enabled: true
# Enable clouddriver-caching's clean up agent to periodically purge old
# clusters and accounts. Set to true when using the Kubernetes provider.
unknown-agent-cleanup-agent:
enabled: false
connectionPools:
default:
# additional connection pool parameters are available here,
# for more detail and to view defaults, see:
# https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
default: true
jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
user: clouddriver_service
password: clouddriver@spinnaker.com
# The following tasks connection pool is optional. At Netflix, clouddriver
# instances pointed to Aurora read replicas have a tasks pool pointed at the
# master. Instances where the default pool is pointed to the master omit a
# separate tasks pool.
tasks:
user: clouddriver_service
jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
password: clouddriver@spinnaker.com
migration:
user: clouddriver_migrate
jdbcUrl: jdbc:mysql://10.0.4.22:3306/clouddriver
password: clouddriver@spinnaker.com
redis:
enabled: false
cache:
enabled: false
scheduler:
enabled: false
taskRepository:
enabled: false
Front50服务
创建数据库
CREATE DATABASE `front50` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW ON `front50`.* TO 'front50_service'@'%' IDENTIFIED BY "front50@spinnaker.com";
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW ON `front50`.* TO 'front50_migrate'@'%' IDENTIFIED BY "front50@spinnaker.com";
修改配置文件
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
spinnaker:
s3:
enabled: false
sql:
enabled: true
connectionPools:
default:
# additional connection pool parameters are available here,
# for more detail and to view defaults, see:
# https://github.com/spinnaker/kork/blob/master/kork-sql/src/main/kotlin/com/netflix/spinnaker/kork/sql/config/ConnectionPoolProperties.kt
default: true
jdbcUrl: jdbc:mysql://10.0.4.22:3306/front50
user: front50_service
password: front50@spinnaker.com
migration:
user: front50_migrate
jdbcUrl: jdbc:mysql://10.0.4.22:3306/front50
password: front50@spinnaker.com
Orca服务
创建数据库
set tx_isolation = 'REPEATABLE-READ';
CREATE SCHEMA `orca` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
GRANT
SELECT, INSERT, UPDATE, DELETE, CREATE, EXECUTE, SHOW VIEW
ON `orca`.*
TO 'orca_service'@'%' IDENTIFIED BY "orca@spinnaker.com" ;
GRANT
SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, LOCK TABLES, EXECUTE, SHOW VIEW
ON `orca`.*
TO 'orca_migrate'@'%' IDENTIFIED BY "orca@spinnaker.com" ;
修改配置文件
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi front50-local.yml
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ vi orca-local.yml
tasks:
useManagedServiceAccounts: true
sql:
enabled: true
connectionPool:
jdbcUrl: jdbc:mysql://10.0.4.22:3306/orca
user: orca_service
password: orca@spinnaker.com
connectionTimeout: 5000
maxLifetime: 30000
# MariaDB-specific:
maxPoolSize: 50
migration:
jdbcUrl: jdbc:mysql://10.0.4.22:3306/orca
user: orca_migrate
password: orca@spinnaker.com
# Ensure we're only using SQL for accessing execution state
executionRepository:
sql:
enabled: true
redis:
enabled: false
# Reporting on active execution metrics will be handled by SQL
monitor:
activeExecutions:
redis: false
# Use SQL for Orca's work queue
# Settings from Netflix and may require adjustment for your environment
# Only validated with AWS Aurora MySQL 5.7
# Please PR if you have success with other databases
keiko:
queue:
sql:
enabled: true
redis:
enabled: false
queue:
zombieCheck:
enabled: true
pendingExecutionService:
sql:
enabled: true
redis:
enabled: false
部署服务
bash-5.0$ hal deploy apply --no-validate
创建Ingress访问web测试
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: spinnaker-service
namespace: spinnaker
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
rules:
- host: spinnaker.xxxx.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: spin-deck
port:
number: 9000
- host: spin-gate.xxxx.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: spin-gate
port:
number: 8084
通过web浏览器访问https://spinnaker.layame.com/ 如下：
注：至于为什么访问https呢？因为我的代理是traefik slb上面做了跳转。当然了这里应该根据自己实际的环境出发！
集成ldap:
至于为什么集成ldap呢？账号安全方面考虑了当然是基于，还有其他的各种方式：Google Groups, GitHub Teams, SAML Roles, or LDAP groups。参照：https://spinnaker.io/docs/setup/other_config/security/。
关于ldap的安装可以参考Kuberneters 搭建openLDAP
首先登陆web管理页面登陆用户：
创建ou-devops
创建inetOrgPerson-zhangpeng
Password设置用户zhangpeng的密码
Commit确认
最终如下：
halyard容器中操作.可能复制命令时候出现异常：Was passed main parameter ' --user-search-base' but no main parameter was defined in your arg class。把代码复制到编辑器处理一下
hal config security authn ldap edit \
--user-search-base 'ou=devops,dc=zy,dc=com' \
--url 'ldap://192.168.1.200:389' \
--user-search-filter 'cn={0}' \
--manager-dn 'cn=admin,dc=zy,dc=com' \
--manager-password '12345678'
hal config security authn ldap enable
bash-5.0$ cd /home/spinnaker/.hal/
bash-5.0$ pwd
/home/spinnaker/.hal
bash-5.0$ cat config
web访问如下：怀疑我traefik 强跳搞的
bash-5.0$ hal deploy apply --no-validate
[root@k8s-master-01 ~]# kubectl get pods -n spinnaker
等待pod起来
进入首页
关于授权
首先登陆ldap web管理页面两个用户组 groupOfUniqueNames yunwenzu devops两个组,根据ldap中组进行授权。
ldap创建用户组与用户
yunweizu-用户zhangpeng
将zhangpeng用户添加到组中：
devop用户组-用户huozhonghao
同理将huozhonghao加入devops组
halyard中配置：
开启ldap security 配置。并增加相关配置：
hal config security authz ldap edit \
--url 'ldap://172.19.252.28:389/dc=xxxx,dc=com' \
--manager-dn 'cn=admin,dc=xxxx,dc=com' \
--manager-password 'xxxxxx' \
--user-dn-pattern 'cn={0}' \
--group-search-base 'ou=devops' \
--group-search-filter 'uniqueMember={0}' \
--group-role-attributes 'cn' \
--user-search-filter 'cn={0}'
hal config security authz edit --type ldap
hal config security authz enable
设置那些用户可以访问集群账户、镜像仓库、应用程序
## 配置yunweizu和group02角色的用户可以使用default这个集群账户
hal config provider kubernetes account edit default \
--add-read-permission yunweizu,group02 \
--add-write-permission yunweizu
## 配置yunweizu角色的用户可以使用my-harbor-registry账户
hal config provider docker-registry account edit my-harbor-registry \
--read-permissions yunweizu \
--write-permissions yunweizu
##更新部署
hal deploy apply
注：group2 copy自泽阳大佬的课程笔记。保留了没有什么实际意义。当然了也可以去掉的......
登陆spinnaker web尝试：
注：用zhangpeng用户建了一个空白的
devops的用户huozhonghao创建一个空白的applications做下测试
就先只看到这里的权限，警告提示告诉你read会所有用户锁定在此应用程序之外。
具体的权限是跟ldap绑定的那么应该是这样的：
1.在ldap管理页面中， 将用户zhangpeng加入devops组
2.spinnaker登陆zhangpeng用户新建一个应用，yunweizu 读写可执行，devops组仅仅可读。
- 创建一个新的用户组platform将huozhonghao用户加入
- spinnaker web登陆huozhonghao用户
嗯 这里也可以看到platform组了 修改一下权限试试，删除一下devops的试试：
增加platform组权限也是失败因为只有read权限，没有writer权限
开启管道权限
halyard容器中操作：
bash-5.0$ pwd
/home/spinnaker/.hal/default/profiles
bash-5.0$ cat /home/spinnaker/.hal/default/profiles/orca-local.yml
tasks:
useManagedServiceAccounts: true
bash-5.0$ cat ~/.hal/default/profiles/settings-local.js
window.spinnakerSettings.feature.managedServiceAccounts = true;
bash-5.0$ hal deploy apply --no-validate
注意：orca-local.yml中的开启。我其实在orca服务中早配置上了！
权限的一些测试
测试一下权限。登陆zhangpeng用户新建一个pipeline zhangpeng
可以发现默认的kubernetes的default account 并可以保存pipeline
huozhonghao用户修改zhangpeng pipeline中的Manifest.嗯没有操作权限
嗯给devops组添加一个read kubernetes account的权限是不是要？否则连account都没有！
bash-5.0$ hal deploy apply --no-validate
[root@k8s-master-01 develop]# kubectl get pods -n spinnaker
等待clouddriver running！
[root@k8s-master-01 develop]#kubectl get svc -n spinnaker
[root@k8s-master-01 develop]# curl -X POST http://172.19.254.33:7003/roles/sync
[root@k8s-master-01 develop]#curl 172.19.254.33:7003/authorize/huozhonghao
read权限依然无法看到accout!
kubernetes default account 添加devops组writer权限：
bash-5.0$ vi config
bash-5.0$ hal deploy apply --no-validate
继续等待clouddriver crunning
嗯再次刷新web登陆huozhonghao用户可以看到kubernetes default account了但是修改Manifest无法writer。验证通过！
安装环境基本完成。其他的步骤后续操作
一些失败的尝试（还是没有成功）
1. 下载Halyard 镜像并启动容器---ctr各种命令的复习
ctr pull
[root@k8s-master-01 ~]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-master-01 ~]# mkdir /root/.hal
参考一下docker时代的启动方式：
docker run -itd --name halyard \
-v /root/.hal:/home/spinnaker/.hal \
-v /root/.kube:/home/spinnaker/.kube \
registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
ctr run
依着葫芦画瓢一下？
ctr run -itd --name halyard \
-v /root/.hal:/home/spinnaker/.hal \
-v /root/.kube:/home/spinnaker/.kube \
registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
中间尝试了很多次各种 ctr命令确实没有搞明白......参考了使用ctr 命令管理 Containerd 容器
我觉得使用containerd安装spinnaker 这真的是可以复习ctr critical命令了
ctr create
[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER IMAGE RUNTIME
halyard registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 io.containerd.runc.v2
ctr t start
[root@k8s-master-01 1.26.6]# ctr t start -d halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK PID STATUS
halyard 1729924 RUNNING
现在问题来了 如何进入容器呢？
ctr tasks exec -t --exec-id
[root@k8s-master-01 1.26.6]# ctr tasks list
TASK PID STATUS
halyard 1729924 RUNNING
[root@k8s-master-01 1.26.6]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $
ctr c rm ctr c kill----读写权限没有搞明白 只能采用挂载本地文件的方式重新搞一波了
嗯哼没有权限？docker的时候可以用root的特权模式进入，这里的ctr也没有找到相关命令。然后就偷懒吧halyard.yml文件copy出来：
true修改为false!
然后挂载文件夹的方式去执行！删除容器重新走一遍流程，走一遍ctr命令
要删除容器应该是先停止？stop？结果不出意外我想错了是kill......当然了ctr t kill --signal 9 halyard强制也很重要
[root@k8s-master-01 1.26.6]# ctr t ls
TASK PID STATUS
halyard 4184764 RUNNING
[root@k8s-master-01 1.26.6]# ctr t kill halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK PID STATUS
halyard 4184764 STOPPED
[root@k8s-master-01 1.26.6]# ctr t ls
TASK PID STATUS
halyard 4184764 STOPPED
[root@k8s-master-01 1.26.6]# ctr c rm halyard
[root@k8s-master-01 1.26.6]# ctr t ls
TASK PID STATUS
[root@k8s-master-01 1.26.6]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 1.26.6]# ctr c ls
CONTAINER IMAGE RUNTIME
halyard registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 io.containerd.runc.v2
[root@k8s-master-01 1.26.6] # ctr t start -d halyard
[root@k8s-master-01 1.26.6] # ctr t ls
TASK PID STATUS
halyard 1729924 RUNNING
[root@k8s-master-01 1.26.6] # ctr tasks exec -t --exec-id 1729924 halyard sh
下载镜像的尝试：
小伙伴们觉得下载镜像应该用下面哪个脚本？用ctr or crictl呢？最终使用镜像的是要kubernetes....应该是用crictl的。 ctr搞了kubernetes集群应用是发现不了镜像的！
#!/bin/bash
S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"
## 下载镜像
function GetImages(){
echo -e "\033[43;34m =====GetImg===== \033[0m"
IMAGES=$( cat tagfile.txt)
for image in ${IMAGES}
do
for node in ${NODES}
do
echo -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
ssh -p 36000 ${node} "crictl pull ${T_REGISTRY}/${image}"
echo -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
ssh -p 36000 ${node} "ctr image tag ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
done
done
for node in ${NODES}
do
echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
ssh -p 36000 ${node} "ctr image ls | grep 'spinnaker-marketplace' "
done
}
GetImages
#!/bin/bash
S_REGISTRY="gcr.io/spinnaker-marketplace"
#T_REGISTRY="registry.cn-beijing.aliyuncs.com/spinnaker-cd"
T_REGISTRY="docker.io/spinnakercd"
NODES="10.0.4.18 10.0.4.49 10.0.4.48 10.0.4.23 10.0.4.47 10.0.4.32"
## 下载镜像
function GetImages(){
echo -e "\033[43;34m =====GetImg===== \033[0m"
IMAGES=$( cat tagfile.txt)
for image in ${IMAGES}
do
for node in ${NODES}
do
echo -e "\033[32m ${node} ---> pull ---> ${image} \033[0m"
ssh -p 36000 ${node} "crictl pull ${T_REGISTRY}/${image}"
echo -e "\033[32m ${node} ---> tag ---> ${image} \033[0m"
ssh -p 36000 ${node} "crictl images ${T_REGISTRY}/${image} ${S_REGISTRY}/${image}"
done
done
for node in ${NODES}
do
echo -e "\033[43;34m =====${node}===镜像信息===== \033[0m"
ssh -p 36000 ${node} "crictl images ls| grep 'spinnaker-marketplace' "
done
}
GetImages
当然了还有一个问题就是 crictl 可以更改镜像名字吗？貌似是不可以的...然后此方式就失败了。
各种失败的尝试-containerd下：
[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER IMAGE RUNTIME
halyard registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 io.containerd.runc.v2
[root@k8s-master-01 .boms]# ctr t start -d halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK PID STATUS
halyard 1775521 RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1729924 halyard sh
/ $ hal config version edit --version local:1.26.6
~ $ cd /home/spinnaker/.hal/
vi config
timezone: America/Los_Angeles
timezone: Asia/Shanghai
hal config storage edit --type s3 --no-validate
hal config security ui edit --override-base-url http://spinnaker.layame.com
hal config security api edit --override-base-url http://spin-gate.layame.com
这都tmd怎么会事情.....要疯了
[root@k8s-master-01 .boms]# ctr t kill --signal 9 halyard
[root@k8s-master-01 .boms]# ctr c rm halyard
[root@k8s-master-01 .boms]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/root/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .boms]# ctr c ls
CONTAINER IMAGE RUNTIME
halyard registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 io.containerd.runc.v2
[root@k8s-master-01 .boms]# ctr t start -d halyard
[root@k8s-master-01 .boms]# ctr t ls
TASK PID STATUS
halyard 1832934 RUNNING
[root@k8s-master-01 .boms]# ctr tasks exec -t --exec-id 1832934 halyard sh
~ $ cd /home/spinnaker/.hal/
~/.hal $ cat config |grep time
timezone: Asia/Shanghai
~/.hal $ cat config |grep s3
persistentStoreType: s3
s3:
s3:
s3Enabled: true
~/.hal $ cat config |grep com
baseUrl: https://api.twilio.com/
overrideBaseUrl: http://spin-gate.layame.com
overrideBaseUrl: http://spinnaker.layame.com
~/.hal $ hal config provider kubernetes enable
~/.hal $ hal config provider kubernetes account add default \
--docker-registries my-harbor-registry \
--context $(kubectl config current-context) \
--service-account true \
--omit-namespaces=kube-system,kube-public \
--provider-version v2 \
--no-validate
至于这个地方的报错 他还是需要w 宿主机 chmod了一下
hal config deploy edit \
--account-name default \
--type distributed \
--location spinnaker
hal config features edit --pipeline-templates true
hal config features edit --artifacts true
hal config features edit --managed-pipeline-templates-v2-ui true
尼玛又疯了！。。。。。。。。。。。。。。。。。分隔符吧 我准备全部都修改好了这些文件了
我又开始怀疑了 一下人生：是不是我的服务器资源不够了？因为我这是kubernetes的master节点，然后呢资源只有4核心8g，我找一个资源多的server测试一下？
先copy一下 .kube下的config
[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.hal
[root@k8s-node-01 home]# mkdir -p /opt/halyard/config
[root@k8s-node-01 home]# mkdir -p /home/spinnaker/.kube
[root@k8s-node-01 home]# crictl pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
Image is up to date for sha256:8673f1670b8768138cd8349b7d9843eb4fd451658227d2e9f02d5fbe454c500d
[root@k8s-node-01 home]# cd /home/spinnaker/.kube
[root@k8s-node-01 .kube]# rz
[root@k8s-node-01 .kube]# ls
config
[root@k8s-node-01 .kube]# ctr image pull registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0
[root@k8s-node-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/home/spinnaker/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-node-01 .boms]# pwd
/home/spinnaker/.hal/.boms
[root@k8s-node-01 .boms]# ls
bom clouddriver deck echo fiat front50 gate igor kayenta monitoring-daemon orca rosco
[root@k8s-node-01 .boms]# cd /opt/halyard/config/
[root@k8s-node-01 config]# cat halyard.yaml
[root@k8s-node-01 ~]# ctr t ls
TASK PID STATUS
[root@k8s-node-01 ~]# ctr t start -d halyard
[root@k8s-node-01 ~]# ctr t ls
TASK PID STATUS
halyard 3910255 RUNNING
[root@k8s-node-01 ~]# ctr tasks exec -t --exec-id 3910255 halyard sh
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
Success
- Edit Spinnaker version
Failure
Validation in Global:
! ERROR Failure writing your halconfig to path
"/home/spinnaker/.hal/config": /home/spinnaker/.hal/config
- Failed to update version.
/ $ hal config version edit --version local:1.26.6
+ Get current deployment
Success
+ Edit Spinnaker version
Success
+ Spinnaker has been configured to update/install version
"local:1.26.6". Deploy this version of Spinnaker with `hal deploy apply`.
/ $ hal config edit --timezone Asia/Shanghai
********又tmd sb了 不知道怎么回事不试了。直接改好配置文件直接启动了！
总结以上失败 执行啥也不行...最后决定直接把docker环境面config文件以及其他制品搞过来试试！
my config文件：
currentDeployment: default
deploymentConfigurations:
- name: default
version: local:1.26.6
providers:
appengine:
enabled: false
accounts: []
aws:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
defaultKeyPairTemplate: '{{name}}-keypair'
defaultRegions:
- name: us-west-2
defaults:
iamRole: BaseIAMRole
ecs:
enabled: false
accounts: []
azure:
enabled: false
accounts: []
bakeryDefaults:
templateFile: azure-linux.json
baseImages: []
dcos:
enabled: false
accounts: []
clusters: []
dockerRegistry:
enabled: true
accounts:
- name: my-harbor-registry
requiredGroupMembership: []
providerVersion: V1
permissions:
READ:
- yunweizu
WRITE:
- yunweizu
address: https://harbor.layame.com
username: zhangpeng
password: xxxx
email: fake.email@spinnaker.io
cacheIntervalSeconds: 30
clientTimeoutMillis: 60000
cacheThreads: 1
paginateSize: 100
sortTagsByDate: false
trackDigests: false
insecureRegistry: false
repositories: []
primaryAccount: my-harbor-registry
google:
enabled: false
accounts: []
bakeryDefaults:
templateFile: gce.json
baseImages: []
zone: us-central1-f
network: default
useInternalIp: false
huaweicloud:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
kubernetes:
enabled: true
accounts:
- name: default
requiredGroupMembership: []
providerVersion: V2
permissions:
READ:
- yunweizu,group02
- devops
WRITE:
- yunweizu
- devops
dockerRegistries:
- accountName: my-harbor-registry
namespaces: []
context: kubernetes-admin@kubernetes
configureImagePullSecrets: true
serviceAccount: true
cacheThreads: 1
namespaces: []
omitNamespaces:
- kube-system
- kube-public
kinds: []
omitKinds: []
customResources: []
cachingPolicies: []
oAuthScopes: []
onlySpinnakerManaged: false
primaryAccount: default
tencentcloud:
enabled: false
accounts: []
bakeryDefaults:
baseImages: []
oracle:
enabled: false
accounts: []
bakeryDefaults:
templateFile: oci.json
baseImages: []
cloudfoundry:
enabled: false
accounts: []
deploymentEnvironment:
size: SMALL
type: Distributed
accountName: default
imageVariant: SLIM
updateVersions: true
consul:
enabled: false
vault:
enabled: false
location: spinnaker
customSizing: {}
sidecars: {}
initContainers: {}
hostAliases: {}
affinity: {}
tolerations: {}
nodeSelectors: {}
gitConfig:
upstreamUser: spinnaker
livenessProbeConfig:
enabled: false
haServices:
clouddriver:
enabled: false
disableClouddriverRoDeck: false
echo:
enabled: false
persistentStorage:
persistentStoreType: s3
azs: {}
gcs:
rootFolder: front50
redis: {}
s3:
rootFolder: front50
oracle: {}
features:
auth: false
fiat: false
chaos: false
entityTags: false
pipelineTemplates: true
artifacts: true
managedPipelineTemplatesV2UI: true
metricStores:
datadog:
enabled: false
tags: []
prometheus:
enabled: false
add_source_metalabels: true
stackdriver:
enabled: false
newrelic:
enabled: false
tags: []
period: 30
enabled: false
notifications:
slack:
enabled: false
twilio:
enabled: false
baseUrl: https://api.twilio.com/
github-status:
enabled: false
timezone: Asia/Shanghai
ci:
jenkins:
enabled: true
masters:
- name: my-jenkins-master-01
permissions: {}
address: https://jenkins.xxxx.com
username: zhangpeng
password: xxxxx
csrf: true
travis:
enabled: false
masters: []
wercker:
enabled: false
masters: []
concourse:
enabled: false
masters: []
gcb:
enabled: false
accounts: []
codebuild:
enabled: false
accounts: []
repository:
artifactory:
enabled: false
searches: []
security:
apiSecurity:
ssl:
enabled: false
overrideBaseUrl: https://spin-gate.xxxx.com
uiSecurity:
ssl:
enabled: false
overrideBaseUrl: https://spinnaker.xxxx.com
authn:
oauth2:
enabled: false
client: {}
resource: {}
userInfoMapping: {}
saml:
enabled: false
userAttributeMapping: {}
ldap:
enabled: true
url: ldap://172.19.252.28:389
userSearchBase: ou=devops,dc=xxxx,dc=com
userSearchFilter: cn={0}
managerDn: cn=admin,dc=xxxx,dc=com
managerPassword: xxxx
x509:
enabled: false
iap:
enabled: false
enabled: true
authz:
groupMembership:
service: LDAP
google:
roleProviderType: GOOGLE
github:
roleProviderType: GITHUB
file:
roleProviderType: FILE
path: /home/spinnaker/.hal/userrole.yml
ldap:
roleProviderType: LDAP
url: ldap://172.19.252.28:389/dc=xxxx,dc=com
managerDn: cn=admin,dc=xxxx,dc=com
managerPassword: xxxx
userDnPattern: cn={0}
groupSearchBase: ou=devops
userSearchFilter: cn={0}
groupSearchFilter: uniqueMember={0}
groupRoleAttributes: cn
enabled: true
artifacts:
bitbucket:
enabled: false
accounts: []
gcs:
enabled: false
accounts: []
oracle:
enabled: false
accounts: []
github:
enabled: true
accounts:
- name: my-github-account
username: zeyangli
token: xxxx
gitlab:
enabled: true
accounts:
- name: my-gitlab-account
token: xxxx
gitrepo:
enabled: false
accounts: []
http:
enabled: false
accounts: []
helm:
enabled: false
accounts: []
s3:
enabled: false
accounts: []
maven:
enabled: false
accounts: []
templates: []
pubsub:
enabled: false
google:
enabled: false
pubsubType: GOOGLE
subscriptions: []
publishers: []
canary:
enabled: false
serviceIntegrations:
- name: google
enabled: false
accounts: []
gcsEnabled: false
stackdriverEnabled: false
- name: prometheus
enabled: false
accounts: []
- name: datadog
enabled: false
accounts: []
- name: signalfx
enabled: false
accounts: []
- name: aws
enabled: false
accounts: []
s3Enabled: false
- name: newrelic
enabled: false
accounts: []
reduxLoggerEnabled: true
defaultJudge: NetflixACAJudge-v1.0
stagesEnabled: true
templatesEnabled: true
showAllConfigsEnabled: true
spinnaker:
extensibility:
plugins: {}
repositories: {}
webhook:
trust:
enabled: false
stats:
enabled: true
endpoint: https://stats.spinnaker.io
instanceId: 01FKDR1B3P8PF35RRC93XTE9AS
deploymentMethod: {}
connectionTimeoutMillis: 3000
readTimeoutMillis: 5000
直接搞过来试一波
上传文件并解压到k8s-master-01节点home目录下
继续
[root@k8s-master-01 .kube]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .kube]# ctr t start -d halyard
[root@k8s-master-01 .kube]# ctr t ls
TASK PID STATUS
halyard 3073271 RUNNING
[root@k8s-master-01 .kube]# ctr tasks exec -t --exec-id 3073271 halyard sh
bash-5.0$ hal deploy apply --no-validate
重新来一遍
[root@k8s-master-01 .kube]# ctr t kill --signal 9 halyard
[root@k8s-master-01 .kube]# ctr c rm halyard
[root@k8s-master-01 .hal]# ctr c create registry.cn-beijing.aliyuncs.com/spinnaker-cd/halyard:1.32.0 halyard --mount type=bind,src=/root/.hal,dst=/home/spinnaker/.hal,options=rbind:row --mount type=bind,src=/home/spinnaker/.kube,dst=/home/spinnaker/.kube,options=rbind:ro --mount type=bind,src=/opt/halyard/config/,dst=/opt/halyard/config/,options=rbind:rw
[root@k8s-master-01 .hal]# ctr t start -d halyard
[root@k8s-master-01 .hal]# ctr t ls
TASK PID STATUS
halyard 3085723 RUNNING
[root@k8s-master-01 .hal]# ctr tasks exec -t --exec-id 3085723 halyard bash
bash-5.0$
算了我放弃了......，containerd的安装方式
总结一下失败以及经验：
- containerd or docker的运行时中都可以在文件夹 /home/spinnaker/.hal/default/service-settings本地写文件的件方式指定image tag，docker环境下还好，containerd方式下crictl 修改镜像标签自己掌握的不是很好！
- containerd命令跟docker还是不一样。启动halyard的方式还是很不好弄，最好的方式还是在一台安装docker的机器上面运行halyard。
- halyard执行脚本复制命令的空格格式问题
- 部署过程中出现数据库地址写错问题...写成了TDSQL-C中的读地址....
