[Argo] 06 - Pipeline based on kinD
This page mainly describes the video: Argo Workflows 101 Workshop 22 Sep 2020
PPT: https://docs.google.com/presentation/d/1ftwxelCtW-onnuxoMMAL-ZAqLOICh9YuAGzgo-dghZQ/edit#slide=id.p3
与kinD不同，K3D有自己的一套命令，所以无法与原生的k8s命令kubectl通用。竟然也需要启动 KinD 创建自己的cluster，then k3d can be available.
Ref: Play with "Argo Workflow" in your local kind cluster.
Ref: Configure Argo to work in kind
S3D系列
Ref: K3d - How to run Kubernetes cluster locally using Rancher K3s
Ref: How to Run Argo Workflows in K3s
Ref: How to Run Argo Workflows with Docker Desktop
貌似 K3d可能比KinD是一个更好的方案？这哥们的观点又变了？
不一定依赖k3s，也可以用原生的k8s。
例如Ubuntu自带的microk8s: https://ubuntu.com/kubernetes/install
前提
Install Kubernetes locally (we recommend use Docker on Desktop + K3D as it supports RBAC): kubectl
HTTPS!!
k3d$ kubectl -n argo port-forward svc/argo-server 2746:2746 Forwarding from 127.0.0.1:2746 -> 2746 Forwarding from [::1]:2746 -> 2746 Handling connection for 2746 Handling connection for 2746 Handling connection for 2746
Did you specify the right host or port?
Ref: The connection to the server localhost:8080 was refused - did you specify the right host or port?
$ kubectl config view apiVersion: v1 clusters: null contexts: null current-context: "" kind: Config preferences: {} users: null
KinD则没有这个问题。创建cluster之后，自动创建了config。
kind$ sudo kind create cluster
Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.25.0) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Set kubectl context to "kind-kind" You can now use your cluster with: kubectl cluster-info --context kind-kind Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/
kind$ sudo kubectl config view
apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://127.0.0.1:43153 name: kind-kind contexts: - context: cluster: kind-kind user: kind-kind name: kind-kind current-context: kind-kind kind: Config preferences: {} users: - name: kind-kind user: client-certificate-data: REDACTED client-key-data: REDACTED
Let's go
查看状态
在namespace argo中创建集群。
kubectl create ns argo kubectl -n argo apply -f https://raw.githubusercontent.com/argoproj/argo/master/manifests/quick-start-postgres.yaml
等待pod全部ready，然后开始Argo UI服务。
kind$ kubectl -n argo get pods --watch kind$ kubectl -n argo port-forward svc/argo-server 2746:2746 Forwarding from 127.0.0.1:2746 -> 2746 Forwarding from [::1]:2746 -> 2746 Handling connection for 2746 Handling connection for 2746 Handling connection for 2746
提交一个任务
创建配置文件并触发 BY UI
metadata: name: wonderful-rhino namespace: argo labels: example: 'true' spec: arguments: parameters: - name: message value: hello argo entrypoint: argosay templates: - name: argosay inputs: parameters: - name: message value: '{{workflow.parameters.message}}' container: name: main image: argoproj/argosay:v2 command: - /argosay args: - echo - '{{inputs.parameters.message}}' ttlStrategy: secondsAfterCompletion: 300 podGC: strategy: OnPodCompletion
先试一下如下这个简单的配置。（稍等一会，PHASE会从pending --> succeeded）
metadata: name: hello namespace: argo spec: entrypoint: main templates: - name: main container: name: pod image: 'docker/whalesay:latest' command: - bash - -c args: - "echo 'Hi Argo Workshop!'"
创建配置文件并触发 BY CMD
UI中也会相应的出现该workflow。
$ argo submit wf.yaml --watch -n argo Name: from-cli-5p6rq Namespace: argo ServiceAccount: unset (will run with the default ServiceAccount) Status: Pending Created: Thu Sep 15 11:13:16 +1000 (now) Progress:
----------------------------------------------------------------------
Name: from-cli-5p6rq Namespace: argo ServiceAccount: unset (will run with the default ServiceAccount) Status: Succeeded Conditions: PodRunning False Completed True Created: Thu Sep 15 11:13:16 +1000 (30 seconds ago) Started: Thu Sep 15 11:13:16 +1000 (30 seconds ago) Finished: Thu Sep 15 11:13:46 +1000 (now) Duration: 30 seconds Progress: 1/1 ResourcesDuration: 15s*(1 cpu),9s*(100Mi memory) STEP TEMPLATE PODNAME DURATION MESSAGE ✔ from-cli-5p6rq main from-cli-5p6rq-main-1919705534 20s
创建配置文件并触发 BY RestAPI
默认RestAPI: https://localhost:2746/apidocs
Ref: Submit parameterized workflow template using REST api in python #8319
import requests argo_token = "" namespace = "argo" template_name = "ni-tmp" workflow_name = "test" headers = {"Authorization": f"{argo_token}"} js = { "resourceKind": "WorkflowTemplate", "namespace": namespace, "resourceName": template_name, "submitOptions": {"labels": f"workflows.argoproj.io/workflow-template={template_name}", "name": workflow_name, }, "arguments": {"parameters": [{"name": "message", "value": "Cheers mate"}]}, # Tried more inputs but same result: # "parameters": [{"name": "message", "value": "Cheers mate"}], # "parameters": ["Cheers mate"], # "arguments": ["Cheers mate"], } response = requests.post( "https://localhost:2746/api/v1/workflows/argo/submit", headers=headers, json=js, verify=False, ) print(response.json()["spec"]["arguments"]) # Returns an empty dict -> {}
https://localhost:2746/api/v1/workflows/argo
1 { 2 "metadata": { 3 "resourceVersion": "13090" 4 }, 5 "items": [ 6 { 7 "metadata": { 8 "name": "python-multi-step-workflow-8lp2x", 9 "generateName": "python-multi-step-workflow-", 10 "namespace": "argo", 11 "uid": "01011f04-6a2d-4cf3-a1c3-c73c5a01ed25", 12 "resourceVersion": "8816", 13 "generation": 8, 14 "creationTimestamp": "2022-09-16T05:45:55Z", 15 "labels": { 16 "workflows.argoproj.io/completed": "true", 17 "workflows.argoproj.io/phase": "Succeeded", 18 "workflows.argoproj.io/workflow-archiving-status": "Archived" 19 }, 20 "annotations": { 21 "workflows.argoproj.io/pod-name-format": "v2" 22 }, 23 "managedFields": [ 24 { 25 "manager": "argo", 26 "operation": "Update", 27 "apiVersion": "argoproj.io/v1alpha1", 28 "time": "2022-09-16T05:45:55Z", 29 "fieldsType": "FieldsV1", 30 "fieldsV1": { 31 "f:metadata": { 32 "f:generateName": {} 33 }, 34 "f:spec": {} 35 } 36 }, 37 { 38 "manager": "workflow-controller", 39 "operation": "Update", 40 "apiVersion": "argoproj.io/v1alpha1", 41 "time": "2022-09-16T05:55:14Z", 42 "fieldsType": "FieldsV1", 43 "fieldsV1": { 44 "f:metadata": { 45 "f:annotations": { 46 ".": {}, 47 "f:workflows.argoproj.io/pod-name-format": {} 48 }, 49 "f:labels": { 50 ".": {}, 51 "f:workflows.argoproj.io/completed": {}, 52 "f:workflows.argoproj.io/phase": {}, 53 "f:workflows.argoproj.io/workflow-archiving-status": {} 54 } 55 }, 56 "f:status": {} 57 } 58 } 59 ] 60 }, 61 "spec": { 62 "templates": [ 63 { 64 "name": "python-multi-step-workflow", 65 "inputs": {}, 66 "outputs": {}, 67 "metadata": {}, 68 "steps": [ 69 [ 70 { 71 "name": "step1", 72 "template": "python-print", 73 "arguments": { 74 "parameters": [ 75 { 76 "name": "message", 77 "value": "hello1............." 78 } 79 ] 80 } 81 } 82 ], 83 [ 84 { 85 "name": "step2", 86 "template": "python-print", 87 "arguments": { 88 "parameters": [ 89 { 90 "name": "message", 91 "value": "hello2............." 92 } 93 ] 94 } 95 } 96 ], 97 [ 98 { 99 "name": "step3", 100 "template": "python-print", 101 "arguments": { 102 "parameters": [ 103 { 104 "name": "message", 105 "value": "hello3............." 106 } 107 ] 108 } 109 } 110 ] 111 ] 112 }, 113 { 114 "name": "python-print", 115 "inputs": { 116 "parameters": [ 117 { 118 "name": "message" 119 } 120 ] 121 }, 122 "outputs": {}, 123 "metadata": {}, 124 "container": { 125 "name": "", 126 "image": "python:3.7", 127 "command": [ 128 "python", 129 "-c", 130 "import sys;print(sys.argv)" 131 ], 132 "args": [ 133 "{{inputs.parameters.message}}" 134 ], 135 "resources": {} 136 } 137 } 138 ], 139 "entrypoint": "python-multi-step-workflow", 140 "arguments": {} 141 }, 142 "status": { 143 "phase": "Succeeded", 144 "startedAt": "2022-09-16T05:45:55Z", 145 "finishedAt": "2022-09-16T05:55:14Z", 146 "progress": "3/3", 147 "nodes": { 148 "python-multi-step-workflow-8lp2x": { 149 "id": "python-multi-step-workflow-8lp2x", 150 "name": "python-multi-step-workflow-8lp2x", 151 "displayName": "python-multi-step-workflow-8lp2x", 152 "type": "Steps", 153 "templateName": "python-multi-step-workflow", 154 "templateScope": "local/python-multi-step-workflow-8lp2x", 155 "phase": "Succeeded", 156 "startedAt": "2022-09-16T05:45:55Z", 157 "finishedAt": "2022-09-16T05:55:14Z", 158 "progress": "3/3", 159 "resourcesDuration": { 160 "cpu": 330, 161 "memory": 211 162 }, 163 "children": [ 164 "python-multi-step-workflow-8lp2x-2215574805" 165 ], 166 "outboundNodes": [ 167 "python-multi-step-workflow-8lp2x-2622593482" 168 ] 169 }, 170 "python-multi-step-workflow-8lp2x-1074402523": { 171 "id": "python-multi-step-workflow-8lp2x-1074402523", 172 "name": "python-multi-step-workflow-8lp2x[2]", 173 "displayName": "[2]", 174 "type": "StepGroup", 175 "templateScope": "local/python-multi-step-workflow-8lp2x", 176 "phase": "Succeeded", 177 "boundaryID": "python-multi-step-workflow-8lp2x", 178 "startedAt": "2022-09-16T05:55:04Z", 179 "finishedAt": "2022-09-16T05:55:14Z", 180 "progress": "1/1", 181 "resourcesDuration": { 182 "cpu": 3, 183 "memory": 2 184 }, 185 "children": [ 186 "python-multi-step-workflow-8lp2x-2622593482" 187 ] 188 }, 189 "python-multi-step-workflow-8lp2x-1938398414": { 190 "id": "python-multi-step-workflow-8lp2x-1938398414", 191 "name": "python-multi-step-workflow-8lp2x[1].step2", 192 "displayName": "step2", 193 "type": "Pod", 194 "templateName": "python-print", 195 "templateScope": "local/python-multi-step-workflow-8lp2x", 196 "phase": "Succeeded", 197 "boundaryID": "python-multi-step-workflow-8lp2x", 198 "startedAt": "2022-09-16T05:54:53Z", 199 "finishedAt": "2022-09-16T05:55:00Z", 200 "progress": "1/1", 201 "resourcesDuration": { 202 "cpu": 3, 203 "memory": 2 204 }, 205 "inputs": { 206 "parameters": [ 207 { 208 "name": "message", 209 "value": "hello2............." 210 } 211 ] 212 }, 213 "outputs": { 214 "artifacts": [ 215 { 216 "name": "main-logs", 217 "s3": { 218 "key": "python-multi-step-workflow-8lp2x/python-multi-step-workflow-8lp2x-python-print-1938398414/main.log" 219 } 220 } 221 ], 222 "exitCode": "0" 223 }, 224 "children": [ 225 "python-multi-step-workflow-8lp2x-1074402523" 226 ], 227 "hostNodeName": "kind-control-plane" 228 }, 229 "python-multi-step-workflow-8lp2x-2148611424": { 230 "id": "python-multi-step-workflow-8lp2x-2148611424", 231 "name": "python-multi-step-workflow-8lp2x[1]", 232 "displayName": "[1]", 233 "type": "StepGroup", 234 "templateScope": "local/python-multi-step-workflow-8lp2x", 235 "phase": "Succeeded", 236 "boundaryID": "python-multi-step-workflow-8lp2x", 237 "startedAt": "2022-09-16T05:54:53Z", 238 "finishedAt": "2022-09-16T05:55:04Z", 239 "progress": "2/2", 240 "resourcesDuration": { 241 "cpu": 6, 242 "memory": 4 243 }, 244 "children": [ 245 "python-multi-step-workflow-8lp2x-1938398414" 246 ] 247 }, 248 "python-multi-step-workflow-8lp2x-2215574805": { 249 "id": "python-multi-step-workflow-8lp2x-2215574805", 250 "name": "python-multi-step-workflow-8lp2x[0]", 251 "displayName": "[0]", 252 "type": "StepGroup", 253 "templateScope": "local/python-multi-step-workflow-8lp2x", 254 "phase": "Succeeded", 255 "boundaryID": "python-multi-step-workflow-8lp2x", 256 "startedAt": "2022-09-16T05:45:55Z", 257 "finishedAt": "2022-09-16T05:54:53Z", 258 "progress": "3/3", 259 "resourcesDuration": { 260 "cpu": 330, 261 "memory": 211 262 }, 263 "children": [ 264 "python-multi-step-workflow-8lp2x-3812343630" 265 ] 266 }, 267 "python-multi-step-workflow-8lp2x-2622593482": { 268 "id": "python-multi-step-workflow-8lp2x-2622593482", 269 "name": "python-multi-step-workflow-8lp2x[2].step3", 270 "displayName": "step3", 271 "type": "Pod", 272 "templateName": "python-print", 273 "templateScope": "local/python-multi-step-workflow-8lp2x", 274 "phase": "Succeeded", 275 "boundaryID": "python-multi-step-workflow-8lp2x", 276 "startedAt": "2022-09-16T05:55:04Z", 277 "finishedAt": "2022-09-16T05:55:10Z", 278 "progress": "1/1", 279 "resourcesDuration": { 280 "cpu": 3, 281 "memory": 2 282 }, 283 "inputs": { 284 "parameters": [ 285 { 286 "name": "message", 287 "value": "hello3............." 288 } 289 ] 290 }, 291 "outputs": { 292 "artifacts": [ 293 { 294 "name": "main-logs", 295 "s3": { 296 "key": "python-multi-step-workflow-8lp2x/python-multi-step-workflow-8lp2x-python-print-2622593482/main.log" 297 } 298 } 299 ], 300 "exitCode": "0" 301 }, 302 "hostNodeName": "kind-control-plane" 303 }, 304 "python-multi-step-workflow-8lp2x-3812343630": { 305 "id": "python-multi-step-workflow-8lp2x-3812343630", 306 "name": "python-multi-step-workflow-8lp2x[0].step1", 307 "displayName": "step1", 308 "type": "Pod", 309 "templateName": "python-print", 310 "templateScope": "local/python-multi-step-workflow-8lp2x", 311 "phase": "Succeeded", 312 "boundaryID": "python-multi-step-workflow-8lp2x", 313 "startedAt": "2022-09-16T05:45:55Z", 314 "finishedAt": "2022-09-16T05:54:44Z", 315 "progress": "1/1", 316 "resourcesDuration": { 317 "cpu": 324, 318 "memory": 207 319 }, 320 "inputs": { 321 "parameters": [ 322 { 323 "name": "message", 324 "value": "hello1............." 325 } 326 ] 327 }, 328 "outputs": { 329 "artifacts": [ 330 { 331 "name": "main-logs", 332 "s3": { 333 "key": "python-multi-step-workflow-8lp2x/python-multi-step-workflow-8lp2x-python-print-3812343630/main.log" 334 } 335 } 336 ], 337 "exitCode": "0" 338 }, 339 "children": [ 340 "python-multi-step-workflow-8lp2x-2148611424" 341 ], 342 "hostNodeName": "kind-control-plane" 343 } 344 }, 345 "conditions": [ 346 { 347 "type": "PodRunning", 348 "status": "False" 349 }, 350 { 351 "type": "Completed", 352 "status": "True" 353 } 354 ], 355 "resourcesDuration": { 356 "cpu": 330, 357 "memory": 211 358 }, 359 "artifactRepositoryRef": { 360 "configMap": "artifact-repositories", 361 "key": "default-v1", 362 "namespace": "argo", 363 "artifactRepository": { 364 "archiveLogs": true, 365 "s3": { 366 "endpoint": "minio:9000", 367 "bucket": "my-bucket", 368 "insecure": true, 369 "accessKeySecret": { 370 "name": "my-minio-cred", 371 "key": "accesskey" 372 }, 373 "secretKeySecret": { 374 "name": "my-minio-cred", 375 "key": "secretkey" 376 } 377 } 378 } 379 }, 380 "artifactGCStatus": { 381 "notSpecified": true 382 } 383 } 384 } 385 ] 386 }
查看该任务
jeffrey@unsw-ThinkPad-T490:kind$ argo get -n argo hello
Name: hello Namespace: argo ServiceAccount: unset (will run with the default ServiceAccount) Status: Succeeded Conditions: PodRunning False Completed True Created: Thu Sep 15 10:28:43 +1000 (52 minutes ago) Started: Thu Sep 15 10:28:43 +1000 (52 minutes ago) Finished: Thu Sep 15 10:29:58 +1000 (51 minutes ago) Duration: 1 minute 15 seconds Progress: 1/1 ResourcesDuration: 24s*(100Mi memory),37s*(1 cpu) STEP TEMPLATE PODNAME DURATION MESSAGE ✔ hello main hello-main-12148776 1m
jeffrey@unsw-ThinkPad-T490:kind$ argo get -n argo hello -o wide
Name: hello Namespace: argo ServiceAccount: unset (will run with the default ServiceAccount) Status: Succeeded Conditions: PodRunning False Completed True Created: Thu Sep 15 10:28:43 +1000 (53 minutes ago) Started: Thu Sep 15 10:28:43 +1000 (53 minutes ago) Finished: Thu Sep 15 10:29:58 +1000 (52 minutes ago) Duration: 1 minute 15 seconds Progress: 1/1 ResourcesDuration: 37s*(1 cpu),24s*(100Mi memory) STEP TEMPLATE PODNAME DURATION ARTIFACTS MESSAGE RESOURCESDURATION NODENAME ✔ hello main hello-main-12148776 1m main-logs 37s*(1 cpu),24s*(100Mi memory) kind-control-plane
jeffrey@unsw-ThinkPad-T490:kind$ argo get -n argo hello -o yaml
metadata: annotations: workflows.argoproj.io/pod-name-format: v2 creationTimestamp: "2022-09-15T00:28:43Z" generation: 4 labels: workflows.argoproj.io/completed: "true" workflows.argoproj.io/creator: system-serviceaccount-argo-argo-server workflows.argoproj.io/phase: Succeeded workflows.argoproj.io/workflow-archiving-status: Archived managedFields: - apiVersion: argoproj.io/v1alpha1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:labels: .: {} f:workflows.argoproj.io/creator: {} f:spec: {} manager: argo operation: Update time: "2022-09-15T00:28:43Z" - apiVersion: argoproj.io/v1alpha1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:workflows.argoproj.io/pod-name-format: {} f:labels: f:workflows.argoproj.io/completed: {} f:workflows.argoproj.io/phase: {} f:workflows.argoproj.io/workflow-archiving-status: {} f:status: {} manager: workflow-controller operation: Update time: "2022-09-15T00:29:58Z" name: hello namespace: argo resourceVersion: "2728" uid: 735668b0-8003-4fd7-9b8a-93c04de3c084 spec: arguments: {} entrypoint: main templates: - container: args: - echo 'Hi Argo Workshop!' command: - bash - -c image: docker/whalesay:latest name: pod resources: {} inputs: {} metadata: {} name: main outputs: {} status: artifactGCStatus: notSpecified: true artifactRepositoryRef: artifactRepository: archiveLogs: true s3: accessKeySecret: key: accesskey name: my-minio-cred bucket: my-bucket endpoint: minio:9000 insecure: true secretKeySecret: key: secretkey name: my-minio-cred configMap: artifact-repositories key: default-v1 namespace: argo conditions: - status: "False" type: PodRunning - status: "True" type: Completed finishedAt: "2022-09-15T00:29:58Z" nodes: hello: displayName: hello finishedAt: "2022-09-15T00:29:49Z" hostNodeName: kind-control-plane id: hello name: hello outputs: artifacts: - name: main-logs s3: key: hello/hello/main.log exitCode: "0" phase: Succeeded progress: 1/1 resourcesDuration: cpu: 37 memory: 24 startedAt: "2022-09-15T00:28:43Z" templateName: main templateScope: local/hello type: Pod phase: Succeeded progress: 1/1 resourcesDuration: cpu: 37 memory: 24 startedAt: "2022-09-15T00:28:43Z"
jeffrey@unsw-ThinkPad-T490:kind$ argo -n argo get @latest
Name: from-cli-5p6rq Namespace: argo ServiceAccount: unset (will run with the default ServiceAccount) Status: Succeeded Conditions: PodRunning False Completed True Created: Thu Sep 15 11:13:16 +1000 (10 minutes ago) Started: Thu Sep 15 11:13:16 +1000 (10 minutes ago) Finished: Thu Sep 15 11:13:46 +1000 (9 minutes ago) Duration: 30 seconds Progress: 1/1 ResourcesDuration: 15s*(1 cpu),9s*(100Mi memory) STEP TEMPLATE PODNAME DURATION MESSAGE ✔ from-cli-5p6rq main from-cli-5p6rq-main-1919705534 20s
jeffrey@unsw-ThinkPad-T490:kind$ argo -n argo logs @latest
from-cli-5p6rq: time="2022-09-15T01:13:35.317Z" level=info msg="capturing logs" argo=true from-cli-5p6rq: hello from-cli-5p6rq: time="2022-09-15T01:13:36.321Z" level=info msg="sub-process exited" argo=true error="<nil>"
Workflow Service Account
投机取巧一下：不适用me，而是使用argo这个默认就有的。
$ kubectl get sa NAME SECRETS AGE argo 0 3h32m argo-server 0 3h32m default 0 3h32m github.com 0 3h32m me 0 27m
然后就稀里糊涂的pass了！
Name: from-cli-rphqp Namespace: argo ServiceAccount: argo Status: Succeeded Conditions: PodRunning False Completed True Created: Thu Sep 15 18:35:15 +1000 (20 seconds ago) Started: Thu Sep 15 18:35:15 +1000 (20 seconds ago) Finished: Thu Sep 15 18:35:35 +1000 (now) Duration: 20 seconds Progress: 1/1 ResourcesDuration: 13s*(1 cpu),8s*(100Mi memory) STEP TEMPLATE PODNAME DURATION MESSAGE ✔ from-cli-rphqp main from-cli-rphqp-main-2187027713 18s
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: workflow-role rules: - apiGroups: - "" resources: - pods verbs: - get - watch - patch - apiGroups: - "" resources: - pods/log verbs: - get - watch
apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: default-workflow-role namespace: argo roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: workflow-role subjects: - kind: ServiceAccount name: default namespace: argo
理解消化一下 cluster role 和 rolebinding。
看样子是非常棒的系列～
Cron Workflows
CronWorkflows are Regular Workflows that run on a schedule. Converting one is easy.
Continue ...
