交付dubbo微服务到k8s (二）

1. 交付dubbo服务提供者

1.1.1 在master机器上安装helm

在master机器 192.168.31.37 上安装helm3.1

wget https://get.helm.sh/helm-v3.1.0-linux-amd64.tar.gz
tar zxvf helm-v3.0.0-linux-amd64.tar.gz 
mv linux-amd64/helm /usr/bin/

1.1.2 配置helm chart

在 master 机器 192.168.31.37上创建helm chart

mkdir /root/helm
helm create dubbo-demo-service

values.yaml

image:
  repository: harbor.od.com/app/dubbo-demo-service 
  tag: master_20200814_0116

templates/dp.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: dubbo-demo-service
  namespace: app
  labels:
    name: dubbo-demo-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dubbo-demo-service
  template:
    metadata:
      labels:
        app: dubbo-demo-service
    spec:
      containers:
      - name: dubbo-demo-service
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }} 
        ports:
        - containerPort: 20880
          protocol: TCP
        env:
        - name: JAR_BALL
          value: dubbo-server.jar
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsUser: 0
      schedulerName: default-scheduler

1.1.3 将helm 的chart拷贝到 `192.168.31.200:/data/jenkins/jenkins_home/helm`

在 192.168.31.200:/data/jenkins/jenkins_home下创建helm目录，将chart从192.168.31.37拷贝到这个目录下

将 master下的 ~/.kube/config 拷贝到helm目录下

tree helm/

1597413680252

1.2 配置参数

1.2.1 设置变量 app_name和image_name

app_name: app名字

变量名: app_name
变量的默认值: dubbo-demo-service

image_name: 镜像名字

变量名: image_name	
变量的默认值: app/dubbo-demo-service

1.2.2 设置变量 git_repo 和 git_version

git_repo: 代码的git地址

变量名: git_repo
变量的默认值: https://gitee.com/chen1219_1/dubbo-demo-service.git

git_version: 拉取代码哪个分支

变量名: git_version
变量的默认值: master

1.2.3 设置变量 add_tag 和 mvn_dir

add_tag: 打包代码后的镜像的tag

变量名: add_tag
变量的默认值: 20200814_0116

mvn_dir: maven打包时候的执行目录

变量名: mvn_dir
变量的默认值: ./

1.2.4 设置变量 target_dir 和 mvn_cmd

target_dir: maven打包完成后生成的jar包所在的目录，一般是在执行目录下生成target目录， jar包在target目录下

变量名: target_dir
变量的默认值: ./dubbo-server/target

mvn_cmd: maven打包时候执行的命令

变量名: mvn_cmd
变量的默认值: mvn clean package -Dmaven.test.skip=true

1.2.5 设置变量 base_image 和 maven

base_image：构建镜像时候所用的底包是哪个镜像

变量名: base_image
变量的选项:
base/jre8:8u112
base/jre7:7u80

maven: 构建镜像时候使用的maven版本

这里的变量的选项要和在 192.168.31.200:/data/jenkins/jenkins_home下的maven的目录是一样的，在打包时候直接是使用这个绝对路径来调用mvn命令进行打包的

变量名: maven
变量的选项:
3.6.3-8u242
3.6.1-8u232

1.3 配置流水线

pipeline {
    agent any

    stages {
        stage('pull') {  //get project code from repo
            steps {
                sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_version}"	
            }
        }
		
		stage('build') {  //exec mvn cmd
            steps {
                sh "cd ${params.app_name}/${env.BUILD_NUMBER} && /var/jenkins_home/maven-${params.maven}/bin/${params.mvn_cmd}"	
				sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
				writeFile file: "${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.od.com/${params.base_image}
				ADD ${params.target_dir}/project_dir /opt/project_dir"""
                sh "cd ${params.app_name}/${env.BUILD_NUMBER} && docker build --no-cache . -t harbor.od.com/${params.image_name}:${params.git_version}_${params.add_tag}  && docker push harbor.od.com/${params.image_name}:${params.git_version}_${params.add_tag}"	
            }
        }
		
		
		stage('deploy') {  //build image and push to registry
            steps {
				sh "helm --kubeconfig /var/jenkins_home/helm/kube_config upgrade dubbo-demo-service -n app --set image.tag=${params.git_version}_${params.add_tag} --install /var/jenkins_home/helm/dubbo-demo-service"
            }
        }
		
    }
}

1.4 配置jenkins参数的值

1.5 新建harbor项目目录

1.6 查看流水线运行状态

1.7 查看harbor仓库

1.8 查看k8s 中是否自动部署成功

1.9 在zookeeper上查看是否注册成功

/opt/zookeeper-3.4.14/bin/zkCli.sh
ls /

2. dubbo-monitor工具

源码： https://github.com/Jeromefromcn/dubbo-monitor

mkdir /root/dubbo-monitor
cd /root/dubbo-monitor
git clone https://gitee.com/chen1219/dubbo-monitor.git

2.1 修改如下配置

192.168.31.37 上操作

/root/dubbo-monitor/dubbo-monitor/dubbo-monitor-simple/conf/dubbo_origin.properties

dubbo.application.name=dubbo-monitor
dubbo.application.owner=Java_winner
dubbo.registry.address=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics

说明： dubbo.registry.address / dubbo.protocol.port 参数参考代码https://gitee.com/chen1219_1/dashboard/projects/chen1219_1/dubbo-demo-service/blob/master/dubbo-server/src/main/java/config.properties

/root/dubbo-monitor/dubbo-monitor/dubbo-monitor-simple/bin/start.sh

#!/bin/bash
sed -e "s/{ZOOKEEPER_ADDRESS}/$ZOOKEEPER_ADDRESS/g" /dubbo-monitor-simple/conf/dubbo_origin.properties > /dubbo-monitor-simple/conf/dubbo.properties
cd `dirname $0`
BIN_DIR=`pwd`
cd ..
DEPLOY_DIR=`pwd`
CONF_DIR=$DEPLOY_DIR/conf

SERVER_NAME=`sed '/dubbo.application.name/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
SERVER_PROTOCOL=`sed '/dubbo.protocol.name/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
SERVER_PORT=`sed '/dubbo.protocol.port/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`
LOGS_FILE=`sed '/dubbo.log4j.file/!d;s/.*=//' conf/dubbo.properties | tr -d '\r'`

if [ -z "$SERVER_NAME" ]; then
    SERVER_NAME=`hostname`
fi

PIDS=`ps -f | grep java | grep "$CONF_DIR" |awk '{print $2}'`
if [ -n "$PIDS" ]; then
    echo "ERROR: The $SERVER_NAME already started!"
    echo "PID: $PIDS"
    exit 1
fi

if [ -n "$SERVER_PORT" ]; then
    SERVER_PORT_COUNT=`netstat -tln | grep $SERVER_PORT | wc -l`
    if [ $SERVER_PORT_COUNT -gt 0 ]; then
        echo "ERROR: The $SERVER_NAME port $SERVER_PORT already used!"
        exit 1
    fi
fi

LOGS_DIR=""
if [ -n "$LOGS_FILE" ]; then
    LOGS_DIR=`dirname $LOGS_FILE`
else
    LOGS_DIR=$DEPLOY_DIR/logs
fi
if [ ! -d $LOGS_DIR ]; then
    mkdir $LOGS_DIR
fi
STDOUT_FILE=$LOGS_DIR/stdout.log

LIB_DIR=$DEPLOY_DIR/lib
LIB_JARS=`ls $LIB_DIR|grep .jar|awk '{print "'$LIB_DIR'/"$0}'|tr "\n" ":"`

JAVA_OPTS=" -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true "
JAVA_DEBUG_OPTS=""
if [ "$1" = "debug" ]; then
    JAVA_DEBUG_OPTS=" -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n "
fi
JAVA_JMX_OPTS=""
if [ "$1" = "jmx" ]; then
    JAVA_JMX_OPTS=" -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false "
fi
JAVA_MEM_OPTS=""
BITS=`java -version 2>&1 | grep -i 64-bit`
if [ -n "$BITS" ]; then
    JAVA_MEM_OPTS=" -server -Xmx128m -Xms128m -Xmn32m -XX:PermSize=16m -Xss256k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:LargePageSizeInBytes=128m -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 "
else
    JAVA_MEM_OPTS=" -server -Xms128m -Xmx128m -XX:PermSize=16m -XX:SurvivorRatio=2 -XX:+UseParallelGC "
fi

echo -e "Starting the $SERVER_NAME ...\c"
exec java $JAVA_OPTS $JAVA_MEM_OPTS $JAVA_DEBUG_OPTS $JAVA_JMX_OPTS -classpath $CONF_DIR:$LIB_JARS com.alibaba.dubbo.container.Main > $STDOUT_FILE 2>&1

scp -r /root/dubbo-monitor/dubbo-monitor/ 192.168.31.200:/data/dockerfile/

2.2 执行Dockerfile

192.168.31.200 上操作

cd /data/dockerfile/dubbo-monitor/
docker build . --no-cache -t harbor.od.com/infra/dubbo-monitor:latest
docker push harbor.od.com/infra/dubbo-monitor:latest

2.3 准备证书

mkdir https
cd https
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=dubbo/0=dubbo"
kubectl create secret tls tls-secret --key tls.key --cert tls.crt

2.4 准备资源

dubbo-monitor-deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: dubbo-monitor
  namespace: infra
  labels:
    name: dubbo-monitor
spec:
  replicas: 1
  selector: 
    matchLabels:
      name: dubbo-monitor
  template:
    metadata:
      labels:
        app: dubbo-monitor
        name: dubbo-monitor
    spec:
      containers:
      - name: dubbo-monitor
        image: harbor.od.com/infra/dubbo-monitor:latest
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 20880
          protocol: TCP
      imagePullSecrets: 
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsUser: 0

dubbo-monitor-svc.yaml

kind: Service
apiVersion: v1
metadata:
  name: dubbo-monitor
  namespace: infra
spec:
  ports:
  - protocol: TCP
    port: 8080
    port: 8080
  selector:
    app: dubbo-monitor

dubbo-monitor-ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-monitor
  namespace: infra
  annotations:
    kubernetes.io/ingress.class: intranet4public
spec:
  tls:
    - hosts: 
      - dubbo-monitor.od.com
      secretName: tls-secret
  rules:
  - host: dubbo-monitor.od.com
    http:
      paths:
      - path: /
        backend:
          serviceName: dubbo-monitor
          servicePort: 8080

dubbo-monitor-configmap.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: dubbo-monitor-cm
  namespace: infra
data:
  dubbo.properties: |
     dubbo.container=log4j,spring,registry,jetty
     dubbo.application.name=dubbo-monitor
     dubbo.application.owner=alibaba
     dubbo.registry.address=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181
     dubbo.protocol.port=20880
     dubbo.jetty.port=8080
     dubbo.jetty.directory=/dubbo-monitor-simple/monitor
     dubbo.charts.directory=/dubbo-monitor-simple/charts
     dubbo.statistics.directory=/dubbo-monitor-simple/statistics
     dubbo.log4j.file=logs/dubbo-monitor-simple.log
     dubbo.log4j.level=WARN

2.5 部署资源

kubectl apply -f dubbo-monitor-deployment.yaml
kubectl apply -f dubbo-monitor-svc.yaml
kubectl apply -f dubbo-monitor-ingress.yaml
kubectl apply -f dubbo-monitor-configmap.yaml

2.6 解析ip地址并访问

C:\Windows\System32\drivers\etc\hosts

192.168.31.40 dubbo-monitor.od.com

浏览器输入https://dubbo-monitor.od.com/

3. 交付dubbo服务消费者

3.1 配置helm chart

helm create dubbo-demo-service

values.yaml

image:
  repository: harbor.od.com/app/dubbo-demo-consumer 
  tag: master_20200815_2242

template/dp.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: dubbo-demo-consumer
  namespace: app
  labels:
    name: dubbo-demo-consumer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: dubbo-demo-consumer
  template:
    metadata:
      labels:
        app: dubbo-demo-consumer
    spec:
      containers:
      - name: dubbo-demo-consumer
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 20880
          protocol: TCP
        env:
        - name: JAR_BALL
          value: dubbo-client.jar
        imagePullPolicy: IfNotPresent
      imagePullSecrets:
      - name: harbor
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      securityContext:
        runAsUser: 0
      schedulerName: default-scheduler

templates/svc.yaml

kind: Service
apiVersion: v1
metadata:
  name: dubbo-demo-consumer
  namespace: app
spec:
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
  selector:
    app: dubbo-demo-consumer

templates/ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: dubbo-demo-consumer
  namespace: app
  annotations:
    kubernetes.io/ingress.class: intranet4public
spec:
  rules:
  - host: demo.od.com
    http:
      paths:
      - path: /
        backend:
          serviceName: dubbo-demo-consumer
          servicePort: 8080

将helm 的chart拷贝到 `192.168.31.200:/data/jenkins/jenkins_home/helm

scp -r /root/helm/dubbo-demo-consumer/ hdss200:/data/jenkins/jenkins_home/helm/

3.2 配置参数

变量名和dubbo-demo的一样，变量的默认值有所改变

app_name: dubbo-demo-consumer
image_name: app/dubbo-demo-consumer
git_repo: git@gitee.com:chen1219_1/dubbo-demo-web.git
git_version: master
add_tag: 20200815_2242
mvn_dir: ./
target_dir: ./dubbo-client/target
mvn_cmd: mvn clean package -Dmaven.test.skip=true
base_image: 
	base/jre8:8u112
	base/jre7:7u80
maven: 
	3.6.3-8u242
	3.6.1-8u232

3.3 配置编译和部署命令

pipeline {
    agent any

    stages {
        stage('pull') {  //get project code from repo
            steps {
                sh "git clone ${params.git_repo} ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.app_name}/${env.BUILD_NUMBER} && git checkout ${params.git_version}"	
            }
        }
		
		stage('build') {  //exec mvn cmd
            steps {
                sh "cd ${params.app_name}/${env.BUILD_NUMBER} && /var/jenkins_home/maven-${params.maven}/bin/${params.mvn_cmd}"	
				sh "cd ${params.app_name}/${env.BUILD_NUMBER} && cd ${params.target_dir} && mkdir project_dir && mv *.jar ./project_dir"
				writeFile file: "${params.app_name}/${env.BUILD_NUMBER}/Dockerfile", text: """FROM harbor.od.com/${params.base_image}
				ADD ${params.target_dir}/project_dir /opt/project_dir"""
                sh "cd ${params.app_name}/${env.BUILD_NUMBER} && docker build --no-cache . -t harbor.od.com/${params.image_name}:${params.git_version}_${params.add_tag}  && docker push harbor.od.com/${params.image_name}:${params.git_version}_${params.add_tag}"	
            }
        }
		
		
 		stage('deploy') {  //build image and push to registry
             steps {
 				sh "helm --kubeconfig /var/jenkins_home/helm/kube_config upgrade dubbo-demo-consumer -n app --set image.tag=${params.git_version}_${params.add_tag} --install /var/jenkins_home/helm/dubbo-demo-consumer"
             }
         }
		
    }
}

3.4 执行流水线

3.5 解析ip地址并访问

C:\Windows\System32\drivers\etc\hosts

192.168.31.40 dubbo-monitor.od.com

浏览器输入http://demo.od.com/

直接访问是访问失败的，浏览器中需要传参数 http://demo.od.com/hello

hello 后面的输出取决于name变量的赋值，比如说 http://demo.od.com/hello?name=world

posted @ 2020-08-10 00:38 cjw1219 阅读(457) 评论(0) 编辑收藏举报

刷新页面返回顶部

cjw1219

罗马不是一天建成的

交付dubbo微服务到k8s (二）

1. 交付dubbo服务提供者

1.1.1 在master机器上安装helm

1.1.2 配置helm chart

1.1.3 将helm 的chart拷贝到 `192.168.31.200:/data/jenkins/jenkins_home/helm`

1.2 配置参数

1.2.1 设置变量 app_name和image_name

1.2.2 设置变量 git_repo 和 git_version

1.2.3 设置变量 add_tag 和 mvn_dir

1.2.4 设置变量 target_dir 和 mvn_cmd

1.2.5 设置变量 base_image 和 maven

1.3 配置流水线

1.4 配置jenkins参数的值

1.5 新建harbor项目目录

1.6 查看流水线运行状态

1.7 查看harbor仓库

1.8 查看k8s 中是否自动部署成功

1.9 在zookeeper上查看是否注册成功

2. dubbo-monitor工具

2.1 修改如下配置

2.2 执行Dockerfile

2.3 准备证书

2.4 准备资源

2.5 部署资源

2.6 解析ip地址并访问

3. 交付dubbo服务消费者

3.1 配置helm chart

3.2 配置参数

3.3 配置编译和部署命令

3.4 执行流水线

3.5 解析ip地址并访问

公告

cjw1219

罗马不是一天建成的

交付dubbo微服务到k8s (二）

1. 交付dubbo服务提供者

1.1.1 在master机器上安装helm

1.1.2 配置helm chart

1.1.3 将helm 的chart拷贝到 192.168.31.200:/data/jenkins/jenkins_home/helm

1.2 配置参数

1.2.1 设置变量 app_name和image_name

1.2.2 设置变量 git_repo 和 git_version

1.2.3 设置变量 add_tag 和 mvn_dir

1.2.4 设置变量 target_dir 和 mvn_cmd

1.2.5 设置变量 base_image 和 maven

1.3 配置流水线

1.4 配置jenkins参数的值

1.5 新建harbor项目目录

1.6 查看流水线运行状态

1.7 查看harbor仓库

1.8 查看k8s 中是否自动部署成功

1.9 在zookeeper上查看是否注册成功

2. dubbo-monitor工具

2.1 修改如下配置

2.2 执行Dockerfile

2.3 准备证书

2.4 准备资源

2.5 部署资源

2.6 解析ip地址并访问

3. 交付dubbo服务消费者

3.1 配置helm chart

3.2 配置参数

3.3 配置编译和部署命令

3.4 执行流水线

3.5 解析ip地址并访问

公告

1.1.3 将helm 的chart拷贝到 `192.168.31.200:/data/jenkins/jenkins_home/helm`