streamsets Executors 说明

执行程序阶段在收到事件时触发任务。执行者不会写入或存储事件。
将执行程序用作事件流中数据流触发器的一部分,以执行事件驱动的与管道相关的任务,例如在目标关闭时移动完全写入的文件。

可以使用的executors

  • Amazon S3 - Creates new Amazon S3 objects for the specified content, copies objects within a bucket, or adds tags to existing Amazon S3 objects.
  • Email - Sends custom email to the configured recipients upon receiving an event.
  • HDFS File Metadata - Changes file metadata, creates an empty file, or removes a file or directory in HDFS or a local file system upon receiving an event record.
  • Hive Query - Runs user-defined Hive or Impala queries upon receiving an event record.
  • JDBC Query - Runs a user-defined SQL query upon receiving an event record.
  • MapR FS File Metadata - Changes file metadata, creates an empty file, or removes a file or directory in MapR FS upon receiving an event record.
  • MapReduce - Starts the specified MapReduce job upon receiving an event record.
  • Pipeline Finisher - Stops and transitions the pipeline to a Finished state upon receiving an event record.
  • Shell - Executes a shell script upon receiving an event record.
  • Spark - Starts the specified Spark application upon receiving an event record.

参考资料

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Executors/Executors-overview.html#concept_stt_2lk_fx

posted on 2018-08-21 15:48  荣锋亮  阅读(456)  评论(0编辑  收藏  举报

导航