Azkaban安装-3.72.0
安装
1、下载tar包
https://github.com/azkaban/azkaban/archive/3.72.0.tar.gz
2、解压
tar -xvf azkaban-3.72.0.tar.gz -C /opt/dsfsadf
3、安装依赖
yum install -y gcc-c++ git
4、进入解压家目录，编译执行
-x test 表示跳过测试
./gradlew build installDist -x test
若编译时失败，重新编译执行：
./gradlew clean build installDist -x test
5、配置MySQL
CREATE DATABASE azkaban;
use azkaban;
CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban';
GRANT ALL ON azkaban.* to 'azkaban'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
source /opt/azkaban-3.72.0/azkaban-db/build/sql/create-all-sql-0.1.0-SNAPSHOT.sql；
6、copy 到指定目录
解压
[root@localhost azkaban-3.73.1]# cp -a azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz ../azkaban
[root@localhost azkaban-3.73.1]# cp -a azkaban-exec-server/build/distributions/azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz ../azkaban
[root@localhost azkaban-3.73.1]# cp -a azkaban-solo-server/build/distributions/azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz ../azkaban
重命名
[root@localhost azkaban]# mv azkaban-exec-server-0.1.0-SNAPSHOT/ azkaban-exec-server
[root@localhost azkaban]# mv azkaban-web-server-0.1.0-SNAPSHOT/ azkaban-web-server
[root@localhost azkaban]# mv azkaban-solo-server-0.1.0-SNAPSHOT/ azkaban-solo-server
7、配置web
进入azkaban web 目录下
cd azkaban-web-server
生成keystore 注意开始和结束的时候输入的密码，其他按提示输入即可 我这里都是azkaban
keytool -keystore keystore -alias jetty -genkey -keyalg RSA
修改conf/azkaban.properties
Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/root/azkaban/azkaban-web-server/conf/azkaban-users.xml
Loader for projects
executor.global.properties=/root/azkaban/azkaban-web-server/conf/global.properties
azkaban.project.dir=projects
Velocity dev mode
velocity.dev.mode=false
Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081
Azkaban Executor settings
mail settings
mail.sender=
mail.host=
User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
enduser -> myazkabanhost:443 -> proxy -> localhost:8081
when this parameters set then these parameters are used to generate email links.
if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
azkaban.webserver.external_hostname=myazkabanhost.com
azkaban.webserver.external_ssl_port=443
azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
JMX stats
jetty.connector.stats=true
executor.connector.stats=true
Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
Multiple Executor
azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.filters=StaticRemainingFlowSize,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1
web端这里：
azkaban.jobtype.plugin.dir=/root/azkaban/azkaban-web-server/plugins/jobtypes
8、配置exec
进入azkaban web 目录下
cd azkaban-exec-server
Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/root/azkaban/azkaban-exec-server/conf/azkaban-users.xml
Loader for projects
executor.global.properties=/root/azkaban/azkaban-exec-server/conf/global.properties
azkaban.project.dir=projects
Velocity dev mode
velocity.dev.mode=false
Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081
Where the Azkaban web server is located
azkaban.webserver.url=http://localhost:8081
mail settings
mail.sender=
mail.host=
User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
enduser -> myazkabanhost:443 -> proxy -> localhost:8081
when this parameters set then these parameters are used to generate email links.
if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
azkaban.webserver.external_hostname=myazkabanhost.com
azkaban.webserver.external_ssl_port=443
azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
JMX stats
jetty.connector.stats=true
executor.connector.stats=true
Azkaban plugin settings
azkaban.jobtype.plugin.dir=/root/azkaban/azkaban-exec-server/plugins/jobtypes
Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
Azkaban Executor settings
executor.maxThreads=50
executor.flow.threads=30
9、启动
查询mysql保持配置一致
select * from executors;
执行exec
./bin/start-exec.sh
执行下面命令激活状态（不可直接修改数据库，jvm没有修改）
curl http://localhost:84654/executor?action=activate
curl -G "localhost:$(<./executor.port)/executor?action=activate" && echo
查看状态
curl http://localhost:59583/executor?action=getStatus
执行web
./bin/start-web.sh
10、查看页面
http://192.168.174.132:8081/index
11、测试
type = command
command = echo 'all flow end'
问题
问题1：
查看log日志发现
azkaban执行错误 Job failed, Cannot request memory (Xms 0 kb, Xmx 0 kb) from system for job
azkaban默认需要3G的内存，剩余内存不足则会报异常。
解决方案：
配置文件，设置（或者增加服务器内存）
azkaban-web-server-2.7.0/plugins/jobtypes/commonprivate.properties
memCheck.enabled=false
问题2：
ERROR [FlowTriggerScheduler] [Azkaban] Unable to get scheduled flow triggers
删掉executor选择器的过滤条件：MinimumFreeMemory
问题3：（遵循启动顺序就好了）
启动顺序不可错，不能直接修改mysql数据库，要通过命令传过去
因为数据库修改了，但是jvm内存的数据没有修改
update executors set active = 1
