[Kubeflow] 02 - Standalone Mode
Ref: Kubeflow Pipelines standalone
其实我关心的是：Local Deployment
Release Schedule
- The full Kubeflow is released quarterly. It has a significant delay in receiving Kubeflow Pipelines updates.
| Kubeflow Version | Kubeflow Pipelines Version |
|---|---|
| 0.7.0 | 0.1.31 |
| 1.0.0 | 0.2.0 |
| 1.0.2 | 0.2.5 |
| 1.1.0 | 1.0.0 |
| 1.2.0 | 1.0.4 |
| 1.3.0 | 1.5.0 |
| 1.4.0 | 1.7.0 |
Note: Google Cloud, AWS, and IBM Cloud have supported Kubeflow Pipelines 1.0.0 with multi-user separation. Other platforms might not be up-to-date for now, refer to this GitHub issue for status.
kind, K3s, K3ai
Ref: [K8S] 01 - MiniKube or K3S
This guide shows how to deploy Kubeflow Pipelines standalone on a local Kubernetes cluster using:
-
- kind
- K3s
- K3s on Windows Subsystem for Linux (WSL)
- K3ai [alpha]
注意：两者都是为了方便测试 kubernetes 集群,切不可用于生产环境。所以造成SaaS还是得用SageMaker等cloud solution。
Kind 顾名思义 Kubernetes in docker，是一个使用 docker 容器在本地运行 Kubernetes 集群的工具。其本身就是为了测试 Kubernetes 而设计，所以天生就和 CI 紧密关联，广泛应用于各种云原生项目的 CI 中，同时因为其可以快速拉起集群和操作简单，深受开发者喜爱，可谓是“有 Kind 不思 Minikube”。
Kind 使用 kubeadm 进行集群的创建，内部使用 containerd 运行组件容器，可以通过指定配置文件 config.yaml 来拉起相应配置的集群，支持多节点集群，同时也可以把本地的镜像加载到集群中，实现测试镜像无需上传镜像仓库的功能。并且之前国内拉取不到镜像的问题已经解决，直接在 dockerhub 拉取 kindest/node 镜像，镜像中均已包含创建 Kubernetes 集群所需的全部资源，无需再额外下载。
Ref: 使用Kubeflow Pipelines
简直就是chedpipe的影子：https://github.com/liuweibin6566396837/kubeflow-examples/tree/master/mnist_stage
官方的一些例子：https://github.com/kubeflow/examples/tree/master/mnist/web-ui
MiniKF? Kubeflow on kinD.
A fast and easy way to deploy Kubeflow on your laptop
Ref: https://v0-6.kubeflow.org/docs/other-guides/virtual-dev/getting-started-minikf/
Ref: Installing Kubeflow Locally with MiniKF and Vagrant
Ref: 带你玩转kubeflow on Kind：使用安装教程【实践出真知】
Ref: End-to-end Kubeflow installation on your personal computer with Kind【实践出真知】
Ref: How to run Kubernetes locally with Kind
Ref: How to run local multi-node Kubernetes clusters using kind
安装 Go
安装 Docker
安装 Kubectl
安装 Kind
安装 Kubeflow
MicroK8S，Ubuntu 自家的孩子
Ref: https://microk8s.io/?ref=thechiefio
Install: https://charmed-kubeflow.io/docs/install
Issue?
$ kubectl get pods --all-namespaces The connection to the server localhost:8080 was refused - did you specify the right host or port?
Ref: Fixing - connection to the server localhost:8080 was refused-did you specify the right host or port?
提供了一点提示。 还是下面的更有效果。
注意，配置文件可以手动如下生成并导入。
Ref: 使用二进制方式安装 K8S 时使用 kubectl 命令报错：The connection to the server localhost:8080 was refused -...
microk8s config > ~/.kube/config
Ref: MicroK8s basics: MicroK8s installation & simple commands
Addon中没有看到 kubeflow。
$ microk8s.enable kubeflow Addon kubeflow was not found in any repository $ microk8s.status microk8s is running high-availability: no datastore master nodes: 127.0.0.1:19001 datastore standby nodes: none addons: enabled: ha-cluster # (core) Configure high availability on the current node disabled: community # (core) The community addons repository dashboard # (core) The Kubernetes dashboard dns # (core) CoreDNS gpu # (core) Automatic enablement of Nvidia CUDA helm # (core) Helm 2 - the package manager for Kubernetes helm3 # (core) Helm 3 - Kubernetes package manager host-access # (core) Allow Pods connecting to Host services smoothly hostpath-storage # (core) Storage class; allocates storage from host directory ingress # (core) Ingress controller for external access mayastor # (core) OpenEBS MayaStor metallb # (core) Loadbalancer for your Kubernetes cluster metrics-server # (core) K8s Metrics Server for API access to service metrics prometheus # (core) Prometheus operator for monitoring and logging rbac # (core) Role-Based Access Control for authorisation registry # (core) Private image registry exposed on localhost:32000 storage # (core) Alias to hostpath-storage add-on, deprecated
Ref: Add-on: Kubeflow
Note: The add-on to install Kubeflow is no longer the recommended way to get up and running. Instead there is a complete end-to-end tutorial on deploying Kubeflow on MicroK8s now published in the Charmed Kubeflow documentation.
https://towardsdatascience.com/kubeflow-how-to-install-and-launch-kubeflow-on-your-local-machine-e0d7b4f7508f
https://becominghuman.ai/installing-a-kubeflow-on-your-local-machine-16ff89b77020
https://medium.com/@gkkarobia/kubeflow-and-local-deployment-of-kubeflow-b0b14885947
未完待续。。。
