本文主要是介绍azkaban安装与使用,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
1. 下载azkaban3.25的”source code.tar.gz”: https://github.com/azkaban/azkaban/releases
2. 解压缩:azkaban-3.25.0.tar.gz,进入azkaban根目录:cd azkaban-3.25.0
3. 执行命令:./gradlew installDist 生成相关文件
4. 创建一个新的文件夹:mkdir azkaban ,复制以下文件内容到XXX目录下:
cp /azkaban-3.25.0/azkaban-web-server/build/install/azkaban-web-server azkaban
cp /azkaban-3.25.0/azkaban-exec-server/build/install/azkaban-exec-server azkaban
cp /azkaban-3.25.0/azkaban-db/build/install/azkaban-web-server azkaban
5. 在mysql数据库中创建一个数据库:azkaban
进入azkaban数据库:use Azkaban
执行:source /azkaban-3.25.0/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql
6. 创建SSL配置:keytool -keystore keystore -alias jetty -genkey -keyalg RSA
提示中只需填写个6位密码即可,其他信息可填可不填
将生成的keystore文件拷贝至XXX目录下:cpkeystore /XXX/azkaban-web-server
7. 先配置好服务器节点上的时区
先生成时区配置文件Asia/Shanghai,用交互式命令 tzselect 即可
拷贝该时区文件,覆盖系统本地时区配置
cp/usr/share/zoneinfo/Asia/Shanghai /etc/localtime
8. 配置azkaban-web-server的conf下的文件:
配置Azkaban.properties
#AzkabanPersonalization Settings
azkaban.name=Test
azkaban.label=MyLocal Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
#AzkabanUserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
#Loaderfor projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=root
mysql.password=1234
mysql.numconnections=100
#Velocity dev mode
velocity.dev.mode=false
# AzkabanJetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8443
jetty.port=8081
jetty.keystore=keystore
jetty.password=123456
jetty.keypassword=123456
jetty.truststore=keystore
jetty.trustpassword=123456
# AzkabanExecutor settings
executor.port=12321
# mailsettings
mail.sender=
mail.host=
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
配置azkaban-users.xml
<azkaban-users>
<user username="azkaban"password="azkaban" roles="admin" groups="azkaban"/>
<user username="metrics"password="metrics" roles="metrics"/>
<user username="admin"password="admin" roles="admin,metrics" />
<role name="admin"permissions="ADMIN" />
<role name="metrics"permissions="METRICS"/>
</azkaban-users>
配置log4j.properties
log4j.rootLogger=INFO,C
log4j.appender.C=org.apache.log4j.ConsoleAppender
log4j.appender.C.Target=System.err
log4j.appender.C.layout=org.apache.log4j.PatternLayout
log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-ddHH:mm:ss} %-5p %c{1}:%L - %m%n
9. 配置azkaban-exec-server的conf下的文件
配置azkaban.properties
#Azkaban
default.timezone.id=Asia/Shanghai
# AzkabanJobTypes Plugins
azkaban.jobtype.plugin.dir=plugins/jobtypes
#Loaderfor projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
azkaban.native.lib=
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=root
mysql.password=1234
mysql.numconnections=100
# AzkabanExecutor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30
# JMXstats
jetty.connector.stats=true
executor.connector.stats=true
log4j.properties
log4j.rootLogger=INFO,C
log4j.appender.C=org.apache.log4j.ConsoleAppender
log4j.appender.C.Target=System.err
log4j.appender.C.layout=org.apache.log4j.PatternLayout
log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-ddHH:mm:ss} %-5p %c{1}:%L - %m%n
配置/plugins/jobtypes/commonprivate.proterties
# setexecute-as-user
execute.as.user=false
#关闭内存检查,若可用内存空间小于3GB会报错
memCheck.enabled=false
10. 测试
新建一个任务:command.job
vi command.job
#command.job
type=command
command=echo 'hello azkaban'
将command.job打包成zip文件:zipcommand.job
在浏览器中输入url:https://localhost:8443,输入配置好的用户和密码,进入azkaban web client
点击[Create Project],输入项目名和描述
上传command.job.zip
执行流
注:过程中可能遇到的错误:
①若azkaban-web-server和azkaban-exec-server的目录conf无log4j.properties文件则会报以下错误:
Exception: java.lang.StackOverflowError thrown from the UncaughtExceptionHandler in thread "main"
②若/azkaban-exec-server/plugins/jobtypes/下commonprivate.proterties文件未配置,则会报以下错误:
未配置:execute.as.user=false
azkaban.utils.UndefinedPropertyException:Missing required property 'azkaban.native.lib'
atazkaban.utils.Props.getString(Props.java:478)
atazkaban.jobExecutor.ProcessJob.run(ProcessJob.java:155)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:694)
atazkaban.execapp.JobRunner.run(JobRunner.java:544)
atjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
atjava.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
22-06-2017 09:53:16 CST a ERROR - Missing required property'azkaban.native.lib' cause: null
未配置:memCheck.enabled=false
java.lang.Exception: Cannot request memory (Xms 0 kb, Xmx 0 kb) from system for job a
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:86)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:590)
at azkaban.execapp.JobRunner.run(JobRunner.java:443)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
这篇关于azkaban安装与使用的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!