安装azkaban单机版和集群版
一、安装单机版
1、安装git
yum install git
2、拉取azkaban源码
git clone https://github.com/azkaban/azkaban.git
3、开始编译
cd azkaban; 开始编译 ./gradlew build installDist
编译过程很漫长，一直在下载包，可能出现超时的情况，然后再继续编译就好
如果出现JCE报错，则需要手动下载相关的包
testV1_1 FAILED
java.lang.RuntimeException: java.lang.RuntimeException: org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machine
at azkaban.crypto.Crypto.decrypt(Crypto.java:76)
at azkaban.crypto.DecryptionTest.testV1_1(DecryptionTest.java:35)
Caused by:
java.lang.RuntimeException: org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machine
at azkaban.crypto.CryptoV1_1.decrypt(CryptoV1_1.java:57)
at azkaban.crypto.Crypto.decrypt(Crypto.java:74)
... 1 more
Caused by:
org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machine
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.handleInvalidKeyException(StandardPBEByteEncryptor.java:1073)
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.decrypt(StandardPBEByteEncryptor.java:1050)
at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.decrypt(StandardPBEStringEncryptor.java:725)
at azkaban.crypto.CryptoV1_1.decrypt(CryptoV1_1.java:55)
... 2 more
azkaban.crypto.EncryptionTest > testEncryption FAILED
org.jasypt.exceptions.EncryptionOperationNotPossibleException: Encryption raised an exception. A possible cause is you are using strong encryption algorithms and you have not installed the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files in this Java Virtual Machine
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.handleInvalidKeyException(StandardPBEByteEncryptor.java:1073)
at org.jasypt.encryption.pbe.StandardPBEByteEncryptor.encrypt(StandardPBEByteEncryptor.java:924)
at org.jasypt.encryption.pbe.StandardPBEStringEncryptor.encrypt(StandardPBEStringEncryptor.java:642)
at azkaban.crypto.CryptoV1_1.encrypt(CryptoV1_1.java:42)
at azkaban.crypto.Crypto.encrypt(Crypto.java:58)
at azkaban.crypto.EncryptionTest.testEncryption(EncryptionTest.java:28)
解决方法：
下载JCE
https://www.oracle.com/technetwork/cn/java/javase/downloads/jce8-download-2133166-zhs.html
我这边使用的JDK8，包含了JCE所需要的jre8 ，解压出来 ，放到jdk安装目录的 jre/lib/security下面
参考 https://yq.aliyun.com/articles/648399
4、启动服务
cd azkaban-solo-server/build/install/azkaban-solo-server; bin/start-solo.sh
注意，这里不能直接进入到bin目录启动，否则会报找不到database.properties的错误
java.io.IOException: Cannot find 'database.properties' file in /opt/local/azkaban-solo-server/build/install/azkaban-solo-server/bin/sql/database.properties at azkaban.database.AzkabanDatabaseSetup.loadDBProps(AzkabanDatabaseSetup.java:178) at azkaban.database.AzkabanDatabaseSetup.loadTableInfo(AzkabanDatabaseSetup.java:102) at azkaban.database.AzkabanDatabaseUpdater.runDatabaseUpdater(AzkabanDatabaseUpdater.java:82) at azkaban.soloserver.AzkabanSingleServer.start(AzkabanSingleServer.java:93) at azkaban.soloserver.AzkabanSingleServer.main(AzkabanSingleServer.java:58)
参考： https://blog.csdn.net/SUDDEV/article/details/101369524
5、查看服务
访问 http://ip:8081
默认账号密码是 azkaban azkaban
6、停止服务
bin/shutdown-solo.sh
7、如果是离线安装，可以在可以上网的机器编译，然后将编译好后的
azkaban/azkaban-solo-server/build/install/azkaban-solo-server 复制到安装机器上，然后启动即可。
二、安装集群版
1、在上面编译好的目录中，azkaban-solo-server是单机版服务，azkaban-exec-server是后台服务，azkaban-web-server 是网页服务
我们现在需要在上面编译好的文件中，将后台服务和网页服务复制到安装目录
cp -r azkaban/azkaban-exec-server/build/install/azkaban-exec-server /opt/local cp -r azkaban/azkaban-web-server/build/install/azkaban-web-server /opt/local
2、创建azkaban依赖的MySQL数据库
mysql> CREATE DATABASE azkaban;
3、生成初始化MySQL数据库文件
cd azkaban/azkaban-db ../gradlew build installDist
文件在azkaban/azkaban-db/build/sql，放到MySQL执行
4、配置和启动后台服务
cd /opt/local/azkaban-exec-server/conf
vim azkaban.properties，修改MySQL配置、时区
default.timezone.id=Asia/Shanghai executor.port=12321 mysql.port=3306 mysql.host=数据库ip mysql.database=azkaban mysql.user=数据库账号 mysql.password=数据库密码
启动服务
bin/start-exec.sh
5、配置和启动网页服务
cd /opt/local/azkaban-web-server/conf
vim azkaban.properties
default.timezone.id=Asia/Shanghai mysql.port=3306 mysql.host=数据库ip mysql.database=azkaban mysql.user=数据库账号 mysql.password=数据库密码
启动服务 bin/start-web.sh
然后发现报错了，发现没有可用的executor
2020/05/15 15:05:46.300 +0800 INFO [AzkabanWebServer] [main] [Azkaban] ************* loginLoggerObjName is null, m ake sure there is a logger with name azkaban.webapp.servlet.LoginAbstractAzkabanServlet 2020/05/15 15:05:46.301 +0800 INFO [ExecutorManager] [main] [Azkaban] Initializing executors from database. 2020/05/15 15:05:46.302 +0800 ERROR [ExecutorManager] [main] [Azkaban] No active executors found 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] Exception in thread "main" 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] azkaban.executor.ExecutorManagerException: No active executors found 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.executor.ActiveExecutors .setupExecutors(ActiveExecutors.java:52) 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.executor.ExecutorManager .setupExecutors(ExecutorManager.java:192) 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.executor.ExecutorManager .initialize(ExecutorManager.java:127) 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.executor.ExecutorManager .start(ExecutorManager.java:141) 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.webapp.AzkabanWebServer. launch(AzkabanWebServer.java:234) 2020/05/15 15:05:46.303 +0800 ERROR [StdOutErrRedirect] [main] [Azkaban] at azkaban.webapp.AzkabanWebServer. main(AzkabanWebServer.java:227)
我们改一下数据库，激活executor
use azkaban; update azkaban.executors set active=1;
再启动一次服务，即可
对于集群部署，将azkaban-web-server按照上面的步骤部署，然后执行 curl -G "http:ip:port/executor?action=activate" 激活节点，或者去MySQL改数据激活
1、
executor became inactive before setting up the flow
描述：重启web server的时候，无法执行旧的flow，日志报错如上
原因：Azkaban Executor 启动后不会自动激活
解决：
执行命令手动激活executor：
# 在MySQL表中看具体的端口 select * from executors;
curl http://localhost:2663/executor?action=activate
要求，先启动exec，然后激活，再启动web
2、
azkaban.utils.UndefinedPropertyException: Missing required property 'azkaban.native.lib
在执行节点的conf配置文件查看该配置项
azkaban-exec-server/conf
找到该配置项的文件，
azkaban.jobtype.plugin.dir
修改不加载本地原生库
execute.as.user=false
azkaban.native.lib=false
3、
Unable to get scheduled flow triggers
描述：在执行任务的时候，任务始终处于Preparing（等待）状态
原因：web server在执行前会对executor进行一个过滤，我们需要修改这个过滤器规则，通常是内存限制
解决：
修改：azkaban-web-server/conf/azkaban.properties
#azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.filters=StaticRemainingFlowSize,CpuStatus
