在kubernetes集群里集成Apollo配置中心(3)之交付Apollo-portal至Kubernetes集群
1.执行apollo-portal数据库脚本
apollo-portal数据库脚本链接：https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql
在mfyxw10.mfyxw.com主机上操作
(1)将apollo-portal数据库脚本下载回来
[root@mfyxw10 ~]# wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql -O apolloportal.sql
(2)导入apollo-portal脚本
[root@mfyxw10 ~]# mysql -uroot -p < apolloportal.sql
(3)查看是否导入成功
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> show databases; #Apollo-portal的数据库名为：ApolloPortalDB
+--------------------+
| Database |
+--------------------+
| ApolloConfigDB |
| ApolloPortalDB |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)
MariaDB [(none)]> use ApolloPortalDB; #切换至ApolloPortalDB数据库
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [ApolloPortalDB]> show tables; #查看ApolloPortalDB数据库下的数据表
+--------------------------+
| Tables_in_ApolloPortalDB |
+--------------------------+
| App |
| AppNamespace |
| Authorities |
| Consumer |
| ConsumerAudit |
| ConsumerRole |
| ConsumerToken |
| Favorite |
| Permission |
| Role |
| RolePermission |
| ServerConfig |
| UserRole |
| Users |
+--------------------------+
14 rows in set (0.00 sec)
MariaDB [ApolloPortalDB]>
(4)ApolloPortal数据库授权
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apolloportal"@"192.168.80.%" identified by "123456";
MariaDB [(none)]>flush privileges;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]>
(5)更新ApolloPortal数据表ServerConfig的Value
[root@mfyxw10 ~]# mysql -uroot -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 44
Server version: 10.1.45-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> use ApolloPortalDB;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [ApolloPortalDB]> show tables;
+--------------------------+
| Tables_in_ApolloPortalDB |
+--------------------------+
| App |
| AppNamespace |
| Authorities |
| Consumer |
| ConsumerAudit |
| ConsumerRole |
| ConsumerToken |
| Favorite |
| Permission |
| Role |
| RolePermission |
| ServerConfig |
| UserRole |
| Users |
+--------------------------+
14 rows in set (0.01 sec)
MariaDB [ApolloPortalDB]> select * from ServerConfig\G;
*************************** 1. row ***************************
Id: 1
Key: apollo.portal.envs
Value: dev
Comment: 可支持的环境列表
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 2. row ***************************
Id: 2
Key: organizations
Value: [{"orgId":"TEST1","orgName":"样例部门1"},{"orgId":"TEST2","orgName":"样例部门2"}]
Comment: 部门列表
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 3. row ***************************
Id: 3
Key: superAdmin
Value: apollo
Comment: Portal超级管理员
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 4. row ***************************
Id: 4
Key: api.readTimeout
Value: 10000
Comment: http接口read timeout
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 5. row ***************************
Id: 5
Key: consumer.token.salt
Value: someSalt
Comment: consumer token salt
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 6. row ***************************
Id: 6
Key: admin.createPrivateNamespace.switch
Value: true
Comment: 是否允许项目管理员创建私有namespace
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
*************************** 7. row ***************************
Id: 7
Key: configView.memberOnly.envs
Value: pro
Comment: 只对项目成员显示配置信息的环境列表，多个env以英文逗号分隔
IsDeleted:
DataChange_CreatedBy: default
DataChange_CreatedTime: 2020-07-06 10:15:59
DataChange_LastModifiedBy:
DataChange_LastTime: 2020-07-06 10:15:59
7 rows in set (0.00 sec)
ERROR: No query specified
MariaDB [ApolloPortalDB]>
更新前的内容如图片所示
更新后如下图所示
[root@mfyxw10 ~]# mysql -uroot -p
MariaDB [(none)]> use ApolloPortalDB;
MariaDB [ApolloPortalDB]> update ServerConfig set Value = '[{"orgId":"yf1","orgName":"Linux学院"},{"orgId":"yf2","orgName":"云计算学院"},{"orgId":"yf3","orgName":"Python学院"},{"orgId":"yf4","orgName":"大数据学院"}]' where Id = 2;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
2.准备apollo-portal软件
在运维主机mfyxw50.mfyxw.com上操作
(1)下载apollo-portal软件
[root@mfyxw50 ~]# cd /opt/src/
[root@mfyxw50 ~]# wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-portal-1.5.1-github.zip
(2)解压apollo-portal软件至相对应的目录
[root@mfyxw50 ~]# mkdir /data/dockerfile/apollo-portal
[root@mfyxw50 ~]# unzip /opt/src/apollo-portal-1.5.1-github.zip -d /data/dockerfile/apollo-portal
(3)删除apollo-portal目录用不到的文件
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/apollo-portal-1.5.1-sources.jar
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/apollo-portal.conf
[root@mfyxw50 ~]# rm -fr /data/dockerfile/apollo-portal/scripts/shutdown.sh
(4)配置数据库连接串
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/config/application-github.properties << EOF
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
EOF
(5)配置Portal的mate service(此处可以不修改，在configmap处已经引入了)
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/config/apollo-env.properties << EOF
local.meta=http://localhost:8080
dev.meta=http://config.od.com:8080
fat.meta=http://fill-in-fat-meta-server:8080
uat.meta=http://fill-in-uat-meta-server:8080
lpt.meta=${lpt_meta}
pro.meta=http://fill-in-pro-meta-server:8080
EOF
(6)更新apollo-portal的startup.sh文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/scripts/startup.sh << EOF
#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=\$(hostname -i)
# SERVER_URL="http://localhost:\$SERVER_PORT"
SERVER_URL="http://\${APOLLO_PORTAL_SERVICE_NAME}:\${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="\$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="\$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="\$JAVA_OPTS -Dserver.port=\$SERVER_PORT -Dlogging.file=\$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=\$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "\$JAVA_HOME" ]] && [[ -x "\$JAVA_HOME/bin/java" ]]; then
javaexe="\$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=\$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "\$javaexe" ]]; then
version=\$("\$javaexe" -version 2>&1 | awk -F '"' '/version/ {print \$2}')
version=\$(echo "\$version" | awk -F. '{printf("%03d%03d",\$1,\$2);}')
# now version is of format 009003 (9.3.x)
if [ \$version -ge 011000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 010000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ \$version -ge 009000 ]; then
JAVA_OPTS="\$JAVA_OPTS -Xlog:gc*:\$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="\$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="\$JAVA_OPTS -Xloggc:\$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="\$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "\$(date) ==== Starting ==== \n"
cd \`dirname \$0\`/..
chmod 755 \$SERVICE_NAME".jar"
./\$SERVICE_NAME".jar" start
rc=\$?;
if [[ \$rc != 0 ]];
then
echo "\$(date) Failed to start \$SERVICE_NAME.jar, return code: \$rc"
exit \$rc;
fi
tail -f /dev/null
EOF
3.制作Apollo-Portal的Docker镜像
在运维主机(mfyxw50.mfyxw.com)上操作
(1)编写Apollo-Portal的Docker文件
[root@mfyxw50 ~]# cat > /data/dockerfile/apollo-portal/Dockerfile << EOF
FROM harbor.od.com/base/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-portal-\${VERSION}.jar /apollo-portal/apollo-portal.jar
ADD config/ /apollo-portal/config
ADD scripts/ /apollo-portal/scripts
CMD ["/apollo-portal/scripts/startup.sh"]
EOF
(2)制作Apollo-portal镜像
[root@mfyxw50 ~]# cd /data/dockerfile/apollo-portal
[root@mfyxw50 apollo-portal]# docker build . -t harbor.od.com/infra/apollo-portal:v1.5.1
[root@mfyxw50 apollo-portal]# docker login harbor.od.com
[root@mfyxw50 apollo-portal]# docker push harbor.od.com/infra/apollo-portal:v1.5.1
4.解析portal.od.com域名
在mfyxw10.mfyxw.com主机操作
(1)设置域名
[root@mfyxw10 ~]# cat > /var/named/od.com.zone << EOF
\$ORIGIN od.com.
\$TTL 600 ; 10 minutes
@ IN SOA dns.od.com. dnsadmin.od.com. (
;序号请加1，表示比之前版本要新
2020031312 ; serial
10800 ; refresh (3 hours)
900 ; retry (15 minutes)
604800 ; expire (1 week)
86400 ; minimum (1 day)
)
NS dns.od.com.
\$TTL 60 ; 1 minute
dns A 192.168.80.10
harbor A 192.168.80.50 ;添加harbor记录
k8s-yaml A 192.168.80.50
traefik A 192.168.80.100
dashboard A 192.168.80.100
zk1 A 192.168.80.10
zk2 A 192.168.80.20
zk3 A 192.168.80.30
jenkins A 192.168.80.100
dubbo-monitor A 192.168.80.100
demo A 192.168.80.100
mysql A 192.168.80.10
config A 192.168.80.100
portal A 192.168.80.100
EOF
(2)重启DNS服务
[root@mfyxw10 ~]# systemctl restart named
(3)测试解析
[root@mfyxw10 ~]# dig -t A portal.od.com @192.168.80.10 +short
192.168.80.100
5.准备Apollo-portal资源配置文件
在运维主机mfyxw50.mfyxw.com上操作
(1)创建用于存放Apollo-portal资源配置文件目录
[root@mfyxw50 ~]# mkdir -p /data/k8s-yaml/apollo-portal
(2)deployment.yaml文件内容如下：
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-portal/deployment.yaml << EOF
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
labels:
name: apollo-portal
spec:
replicas: 1
selector:
matchLabels:
name: apollo-portal
template:
metadata:
labels:
app: apollo-portal
name: apollo-portal
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-portal-cm
containers:
- name: apollo-portal
image: harbor.od.com/infra/apollo-portal:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-portal/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
EOF
(3)service.yaml文件内容如下：
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-portal/service.yaml << EOF
kind: Service
apiVersion: v1
metadata:
name: apollo-portal
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-portal
clusterIP: None
type: ClusterIP
sessionAffinity: None
EOF
(4)Ingress.yaml文件内容如下：
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-portal/Ingress.yaml << EOF
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
spec:
rules:
- host: portal.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-portal
servicePort: 8080
EOF
(5)ConfigMap.yaml文件内容如下：
[root@mfyxw50 ~]# cat > /data/k8s-yaml/apollo-portal/configmap.yaml << EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-portal-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
app.properties: |
appId=100003173
apollo-env.properties: |
dev.meta=http://config.od.com
EOF
6.应用Apollo-portal资源配置清单
在master节点(mfyxw30.mfyxw.com或mfyxw40.mfyxw.com)任意一台上执行即可
(1)应用Apollo-portal资源配置清单
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/deployment.yaml
deployment.extensions/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/service.yaml
service/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/Ingress.yaml
ingress.extensions/apollo-portal created
[root@mfyxw30 ~]#
[root@mfyxw30 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/configmap.yaml
configmap/apollo-portal-cm created
(2)查看Apollo-portal的pod是否正常启动
[root@mfyxw30 ~]# kubectl get pod -n infra
NAME READY STATUS RESTARTS AGE
apollo-adminservice-5cccf97c64-bhqzb 1/1 Running 1 6h22m
apollo-configservice-5f6555448-7wxsp 1/1 Running 1 6h16m
apollo-portal-57bc86966d-jz5vg 1/1 Running 0 112s
dubbo-monitor-6676dd74cc-9hghb 1/1 Running 13 17d
dubbo-monitor-6676dd74cc-rd86g 1/1 Running 12 17d
jenkins-b99776c69-p6skp 1/1 Running 17 39d
7.使用浏览器portal.od.com登录
Apollo-portal默认的用户为：apollo 密码为：admin 第一次登录后，强烈建议修改密码，在此修改密码为：admin123
