azkaban记录

bug1

跟着尚硅谷学习数仓项目中使用到了azkaban，在启动的过程中需要首先启动全部的server，然后激活，才能够正确启动web进程。
不过启动的步骤有些繁琐，就想着写成sh脚本启动会来得更方便
脚本如下

#!/bin/bash

if [ $# -lt 1 ]
then
    echo "No Args Input..."
    exit ;
fi

case $1 in
"start")
        echo " =================== 启动 azkaban-server ==================="

        ssh hadoop102 "cd /opt/module/azkaban/azkaban-exec/; bin/start-exec.sh; "
        ssh hadoop103 "cd /opt/module/azkaban/azkaban-exec/; bin/start-exec.sh; "
        ssh hadoop104 "cd /opt/module/azkaban/azkaban-exec/; bin/start-exec.sh; "
		
	echo " =================== activating azkaban-server ==================="
	sleep 1
	ssh hadoop102 "curl -G 'hadoop102:12321/executor?action=activate' && echo"
	sleep 1
	ssh hadoop103 "curl -G 'hadoop103:12321/executor?action=activate' && echo"
	sleep 1
	ssh hadoop104 "curl -G 'hadoop104:12321/executor?action=activate' && echo"
		
	echo " =================== 启动 azkaban-web ==================="
	sleep 1
	ssh hadoop102 "cd /opt/module/azkaban/azkaban-web/; bin/start-web.sh;"
;;
"stop")
        echo " =================== 关闭 azkaban-server ==================="

        ssh hadoop102 "cd /opt/module/azkaban/azkaban-exec/; bin/shutdown-exec.sh;"
        ssh hadoop103 "cd /opt/module/azkaban/azkaban-exec/; bin/shutdown-exec.sh;"
        ssh hadoop104 "cd /opt/module/azkaban/azkaban-exec/; bin/shutdown-exec.sh;"
		
	echo " =================== 关机 azkaban-web ==================="
	ssh hadoop102 "cd /opt/module/azkaban/azkaban-web/; bin/shutdown-web.sh;"
;;
*)
    echo "Input Args Error..."
;;
esac

在启动部分中，如果不在curl -G 'hadoop102:12321/executor?action=activate' && echo前添加sleep1的话，会报错curl: (7) couldn't connect to host
我不知道为什么，但是如果把脚本中的指令一行一行输入到命令行窗口中执行，就不会出现这样的问题，我怀疑是执行得太快。。所以让他们sleep一下，慢点执行。
虽然问题解决，但是真是原因还未发现

bug2

在之前的问题解决之后，我企图启动web页面http://hadoop102:8081/，页面报错500了

Privoxy encountered an error while processing your request:
Could not load template file no-server-data or one of its included components.
Please contact your proxy administrator.
If you are the proxy administrator, please put the required file(s)in the (confdir)/templates directory. The location of the (confdir) directory is specified in the main Privoxy config file. (It's typically the Privoxy install directory).

但是问题的原因很蠢，之前访问了github，觉得访问起来太慢，于是启动了全局VPN，于是导致了这样的问题。。

bug3

使用azkaban调度虚拟机上的shell脚本的时候，我指配了指定的executor进行执行，但是显示任务执行失败，报错如下

16-02-2022 22:34:29 CST hdfs_to_ods_log ERROR - Job run failed!
java.lang.IllegalStateException: Process has not yet started.
	at azkaban.jobExecutor.utils.process.AzkabanProcess.checkStarted(AzkabanProcess.java:263)
	at azkaban.jobExecutor.utils.process.AzkabanProcess.getProcessId(AzkabanProcess.java:161)
	at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:314)
	at azkaban.execapp.JobRunner.runJob(JobRunner.java:823)
	at azkaban.execapp.JobRunner.doRun(JobRunner.java:602)
	at azkaban.execapp.JobRunner.run(JobRunner.java:563)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
16-02-2022 22:34:29 CST hdfs_to_ods_log ERROR - Process has not yet started. cause: null
16-02-2022 22:34:29 CST hdfs_to_ods_log INFO - Finishing job hdfs_to_ods_log at 1645022069772 with status FAILED