kestra 试用体验
kestra 官方是提供了基于docker-compose 运行的模版,以下主要是体验下,实际上kestra 提供的一些能力还是很强大的
尤其是强大的插件开发能力
环境准备
- docker-compose 文件
volumes:
postgres-data:
driver: local
kestra-data:
driver: local
services:
postgres:
image: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: kestra
POSTGRES_USER: kestra
POSTGRES_PASSWORD: k3str4
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
interval: 30s
timeout: 10s
retries: 10
kestra:
image: kestra/kestra:latest-full
pull_policy: always
entrypoint: /bin/bash
# Note that this is meant for development only. Refer to the documentation for production deployments of Kestra which runs without a root user.
user: "root"
command:
- -c
- /app/kestra server standalone --worker-thread=128
volumes:
- kestra-data:/app/storage
- /var/run/docker.sock:/var/run/docker.sock
- /tmp/kestra-wd:/tmp/kestra-wd
environment:
KESTRA_CONFIGURATION: |
datasources:
postgres:
url: jdbc:postgresql://postgres:5432/kestra
driverClassName: org.postgresql.Driver
username: kestra
password: k3str4
kestra:
server:
basic-auth:
enabled: false
username: admin
password: kestra
repository:
type: postgres
storage:
type: local
local:
base-path: "/app/storage"
queue:
type: postgres
tasks:
tmp-dir:
path: /tmp/kestra-wd/tmp
url: http://localhost:8080/
ports:
- "8080:8080"
- "8081:8081"
depends_on:
postgres:
condition: service_started
运行&使用
- 启动
docker-compose up -d
- 访问
模式登陆完成,开启任务官方也有一个引导可以帮助我们快速使用kestra(会自动创建相关的组建任务)
效果如下
示例task 定义,可以看到就是一个yaml 文件,包含了输入,任务,以及触发
# Flow declaration with a mandatory unique identifier, a namespace, and an optional description.
# Flow identifier are unique inside a namespace.
id: welcomeKestra
namespace: io.kestra.tour
description: Welcome to Kestra!
# Flow inputs: each input has a name, a type, and an optional default value.
inputs:
# We define one input of name 'user' with a default value 'Data Engineer'
- name: user
type: STRING
defaults: Kestra user
# List of tasks that will be executed one after the other.
# Each task must have an identifier unique for the flow and a type.
# Depending on the type of the task, you may have to pass additional attributes.
tasks:
# This is one of the simplest task: it echos a message in the log, like the 'echo' command.
# The message is passed thanks to the 'format' attribute.
# We use the variable from the 'inputs' : {{ and }} are separator of a Pebble expression in which we can access variables.
- id: hello
type: io.kestra.core.tasks.log.Log
message: Hey there, {{ inputs.user }}!
# This task runs a bash command.
# Here we just 'echo' a message, using again the input variable.
- id: goodbye
type: io.kestra.core.tasks.scripts.Bash
commands:
- echo See you soon, {{ inputs.user }}!
# To trigger the Flow we use the 'triggers' property.
triggers:
# Here we use the 'schedule' trigger to run the flow every minute.
- id: everyMinute
type: io.kestra.core.models.triggers.types.Schedule
cron: "*/1 * * * *"
inputs:
name: Kestra master user
执行信息
说明
kestra 是基于java 开发的,开发框架基于了micronaut,支持丰富的插件开发,同时官方已经提供了不少插件(开发的插件对于包含以来的需要打包为shadow fat jar),对于数据开发是一个不错的选择,当然kestra 是包含企业版的,社区版缺少ha 以及访问控制的能力
参考资料
https://github.com/kestra-io/kestra
https://kestra.io/docs/architecture
https://kestra.io/
https://kestra.io/enterprise
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· DeepSeek “源神”启动!「GitHub 热点速览」
· 我与微信审核的“相爱相杀”看个人小程序副业
· 上周热点回顾(2.17-2.23)
· 如何使用 Uni-app 实现视频聊天(源码,支持安卓、iOS)
· spring官宣接入deepseek,真的太香了~
2022-08-14 一些网络延迟测试工具
2022-08-14 orioledb 集成citus 测试
2022-08-14 orioledb 集成 citus docker 镜像
2021-08-14 nicolaka/netshoot 强大的容器网络问题解决工具
2021-08-14 使用nsenter 解决tcpcollect容器网络捕捉问题
2019-08-14 netflix flamescope 方便的不同时间范围的火焰图查看工具
2018-08-14 graphql cli 开发graphql api flow