airflow学习笔记——sensor

sensor也是airflow的一种operator，用于检测某个条件是否达到。如果条件满足，sensor将会执行成功；如果条件不满足，sensor将会重试，直到超时，task超时的时候状态就位skipped。

下面是常用的几种sensor：

• The FileSensor: Waits for a file or folder to land in a filesystem.
• The S3KeySensor: Waits for a key to be present in a S3 bucket.
• The SqlSensor: Runs a sql statement repeatedly until a criteria is met.
• The HivePartitionSensor: Waits for a partition to show up in Hive.
• The ExternalTaskSensor: Waits for a different DAG or a task in a different DAG to complete for a specific execution date. (Pretty useful that one 🤓 )
• The DateTimeSensor: Waits until the specified datetime (Useful to add some delay to your DAGs)
• The TimeDeltaSensor: Waits for a timedelta after the task’s execution_date + schedule interval (Looks similar to the previous one no?)

参考：Airflow Sensors : What you need to know

以及 How Airbnb Built “Wall” to prevent data bugs