kestra 试用体验

kestra 官方是提供了基于docker-compose 运行的模版，以下主要是体验下，实际上kestra 提供的一些能力还是很强大的
尤其是强大的插件开发能力

环境准备

  • docker-compose 文件
 
volumes:
  postgres-data:
    driver: local
  kestra-data:
    driver: local
 
services:
  postgres:
    image: postgres
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: kestra
      POSTGRES_USER: kestra
      POSTGRES_PASSWORD: k3str4
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      interval: 30s
      timeout: 10s
      retries: 10
 
  kestra:
    image: kestra/kestra:latest-full
    pull_policy: always
    entrypoint: /bin/bash
    # Note that this is meant for development only. Refer to the documentation for production deployments of Kestra which runs without a root user.
    user: "root"
    command:
      - -c
      - /app/kestra server standalone --worker-thread=128
    volumes:
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: |
        datasources:
          postgres:
            url: jdbc:postgresql://postgres:5432/kestra
            driverClassName: org.postgresql.Driver
            username: kestra
            password: k3str4
        kestra:
          server:
            basic-auth:
              enabled: false
              username: admin
              password: kestra
          repository:
            type: postgres
          storage:
            type: local
            local:
              base-path: "/app/storage"
          queue:
            type: postgres
          tasks:
            tmp-dir:
              path: /tmp/kestra-wd/tmp
          url: http://localhost:8080/
    ports:
      - "8080:8080"
      - "8081:8081"
    depends_on:
      postgres:
        condition: service_started

运行&使用

  • 启动
docker-compose up -d
  • 访问

模式登陆完成，开启任务官方也有一个引导可以帮助我们快速使用kestra（会自动创建相关的组建任务）
效果如下

 



示例task 定义,可以看到就是一个yaml 文件，包含了输入，任务，以及触发

 
# Flow declaration with a mandatory unique identifier, a namespace, and an optional description.
# Flow identifier are unique inside a namespace.
id: welcomeKestra
namespace: io.kestra.tour
description: Welcome to Kestra!
 
# Flow inputs: each input has a name, a type, and an optional default value.
inputs:
  # We define one input of name 'user' with a default value 'Data Engineer'
- name: user
  type: STRING
  defaults: Kestra user
 
# List of tasks that will be executed one after the other.
# Each task must have an identifier unique for the flow and a type.
# Depending on the type of the task, you may have to pass additional attributes.
tasks:
  # This is one of the simplest task: it echos a message in the log, like the 'echo' command.
  # The message is passed thanks to the 'format' attribute.
  # We use the variable from the 'inputs' : {{ and }} are separator of a Pebble expression in which we can access variables.
- id: hello
  type: io.kestra.core.tasks.log.Log
  message: Hey there, {{ inputs.user }}!
 
  # This task runs a bash command.
  # Here we just 'echo' a message, using again the input variable.
- id: goodbye
  type: io.kestra.core.tasks.scripts.Bash
  commands:
  - echo See you soon, {{ inputs.user }}!
 
 # To trigger the Flow we use the 'triggers' property.
triggers:
  # Here we use the 'schedule' trigger to run the flow every minute.
    - id: everyMinute
      type: io.kestra.core.models.triggers.types.Schedule
      cron: "*/1 * * * *"
      inputs:
        name: Kestra master user

执行信息

 

说明

kestra 是基于java 开发的，开发框架基于了micronaut，支持丰富的插件开发，同时官方已经提供了不少插件(开发的插件对于包含以来的需要打包为shadow fat jar)，对于数据开发是一个不错的选择，当然kestra 是包含企业版的，社区版缺少ha 以及访问控制的能力

参考资料

https://github.com/kestra-io/kestra
https://kestra.io/docs/architecture
https://kestra.io/
https://kestra.io/enterprise

posted on 2023-08-14 21:12  荣锋亮  阅读(942)  评论(1)  编辑  收藏  举报

导航