本文主要是介绍【Azkaban】 Missing required property ‘azkaban.native.lib‘ cause: null，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
1.使用环境
版本：Azkaban3.X
部署模式：集群
2.问题描述
Azkaban一直执行都没有问题，最近执行job的时候全部都包以下错误
azkaban.utils.UndefinedPropertyException: Missing required property 'azkaban.native.lib'at azkaban.utils.Props.getString(Props.java:454)at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:242)at azkaban.execapp.JobRunner.runJob(JobRunner.java:823)at azkaban.execapp.JobRunner.doRun(JobRunner.java:602)at azkaban.execapp.JobRunner.run(JobRunner.java:563)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
21-04-2021 11:09:11 CST pod-mes-pull-order ERROR - Missing required property 'azkaban.native.lib' cause: null
21-04-2021 11:09:11 CST pod-mes-pull-order INFO - Finishing job pod-mes-pull-order at 1618974551015 with status FAILED3.解决问题过程
3.1根据网上通俗的解决方法
1.查看目录:
conf/azkaban.properties
2.查看选项
azkaban.jobtype.plugin.dir=plugins/jobtypes
3.在配置的目录 plugins/jobtype 下的 commonprivate.properties
# set execute-as-user
execute.as.user=false
azkaban.native.lib=false4.然后重启 azkaban
但是最后还是报同样错误，后来才知道这是Azkaban单机部署解决方式，不适合集群部署，因为集群部署azkaban.native.lib是需要配置，不能配置为false
4.参考集群部署解决
4.1验证azkaban.native.lib路径问题
后来参考了集群部署的方式(Azkaban集群部署的配置)，解决问题的关键是execute-as-user 这个文件
查看plugins/jobtype 下的 commonprivate.properties ，azkaban.native.lib路径是能对应上execute-as-user这个文件
## hadoop security manager setting common to all hadoop jobs
hadoop.security.manager.class=azkaban.security.HadoopSecurityManager_H_2_0## hadoop security related settings# proxy.keytab.location=
# proxy.user=# azkaban.should.proxy=true
# obtain.binary.token=true
# obtain.namenode.token=true
# obtain.jobtracker.token=true# global classpath items for all jobs. e.g. hadoop-core jar, hadoop conf
#jobtype.global.classpath=${hadoop.home}/*,${hadoop.home}/conf# global jvm args for all jobs. e.g. java.io.temp.dir, java.library.path
#jobtype.global.jvm.args=# configs for jobtype security settings
execute.as.user=false
# 后面的路径是你放置execute-as-user这个文件的路径
azkaban.native.lib=/opt/azkaban/azkaban-exec/execute-as-user
# 使用系统用户提交的时候，azkaban默认把它们放入azkaban组
# 所以你需要提前创建好azkaban这个组或者修改为一个已存在的组
azkaban.group.name=azkaban
4.2 重新编译execute-as-user
既然路径是对的上，那问题就还剩下在execute-as-user 这个文件，于是去了官网下载了execute-as-user.c来代替现有的execute-as-user(记得将现在execute-as-user文件保存一份)
execute-as-user.c文件地址：https://github.com/azkaban/azkaban/blob/3.89.0/az-exec-util/src/main/c/execute-as-user.c(记得与官网保持最新的代码)
/** Copyright 2017 LinkedIn Corp.** Licensed under the Apache License, Version 2.0 (the "License"); you may not* use this file except in compliance with the License. You may obtain a copy of* the License at** http://www.apache.org/licenses/LICENSE-2.0** Unless required by applicable law or agreed to in writing, software* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the* License for the specific language governing permissions and limitations under* the License.*/#include <dirent.h>
#include <fcntl.h>
#include <fts.h>
#include <errno.h>
#include <grp.h>
#include <unistd.h>
#include <signal.h>
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#include <sys/stat.h>
#include <sys/mount.h>
#include <sys/types.h>
#include <pwd.h>FILE *LOGFILE = NULL;
FILE *ERRORFILE = NULL;
int SETUID_OPER_FAILED = 10;
int USER_NOT_FOUND = 20;
int INVALID_INPUT = 30;/** Change the real and effective user and group from super user to the specified user** Adopted from:* ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c**/int change_user(char *username, uid_t user, gid_t group) {if (user == getuid() && user == geteuid() &&group == getgid() && group == getegid()) {return 0;}if (initgroups(username, group) != 0) {fprintf(LOGFILE, "Error setting supplementary groups for user %s: %s\n",username, strerror(errno));return SETUID_OPER_FAILED;}if (seteuid(0) != 0) {fprintf(LOGFILE, "unable to reacquire root - %s\n", strerror(errno));fprintf(LOGFILE, "Real: %d:%d; Effective: %d:%d\n",getuid(), getgid(), geteuid(), getegid());return SETUID_OPER_FAILED;}if (setgid(group) != 0) {fprintf(LOGFILE, "unable to set group to %d - %s\n", group,strerror(errno));fprintf(LOGFILE, "Real: %d:%d; Effective: %d:%d\n",getuid(), getgid(), geteuid(), getegid());return SETUID_OPER_FAILED;}if (setuid(user) != 0) {fprintf(LOGFILE, "unable to set user to %d - %s\n", user, strerror(errno));fprintf(LOGFILE, "Real: %d:%d; Effective: %d:%d\n",getuid(), getgid(), geteuid(), getegid());return SETUID_OPER_FAILED;}return 0;
}int main(int argc, char **argv){// set up the logging streamif (!LOGFILE){LOGFILE=stdout;}if (!ERRORFILE){ERRORFILE=stderr;}if (argc < 3) {fprintf(ERRORFILE, "Requires at least 3 variables: ./execute-as-user username command [args]");return INVALID_INPUT;}char *username = argv[1];// gather information about userstruct passwd *user_info = getpwnam(username);if (user_info == NULL){fprintf(LOGFILE, "user does not exist: %s", username);return USER_NOT_FOUND;}// try to change userfprintf(LOGFILE, "Changing user: user: %s, uid: %d, gid: %d\n", username, user_info->pw_uid, user_info->pw_gid);int retval = change_user(username, user_info->pw_uid, user_info->pw_gid);if (retval != 0){fprintf(LOGFILE, "Error changing user to %s\n", username);return SETUID_OPER_FAILED;}// execute the commandchar **user_argv = &argv[2];fprintf(LOGFILE, "user command starting from: %s\n", user_argv[0]);fflush(LOGFILE);retval = execvp(*user_argv, user_argv);fprintf(LOGFILE, "system call return value: %d\n", retval);// sometimes system(cmd) returns 256, which is interpreted to 0, making a failed job a successful job// hence this goofy piece of if statement.if (retval != 0){return 1;}else{return 0;}}创建文件execute-as-user.c，并把上面的代码帖进去，然后上传到服务器/opt/azkaban/azkaban-exec(azkaban.native.lib配置的目录)，然后执行以下命令：
gcc execute-as-user.c -o execute-as-user
chown root execute-as-user
chmod 6050 execute-as-user命令解释：
1.使用 gcc execute-as-user.c -o execute-as-user 命令编译
2.然后使用chown root execute-as-user 和 chmod 6050 execute-as-user 设置权限
设置完权限后，ls -l 看下，这个文件的属性这样的
---Sr-s--- 1 root root 10185 Aug 3 13:02 execute-as-user如果我们要把这个文件复制到其他目录下，就又会变成普通文件，记得复制完了再做一次 chmod 6050 execute-as-user
最后重新executor服务
#Executor服务关闭
sh /opt/azkaban/azkaban-exec/bin/shutdown-exec.sh
#Executor服务启动
sh /opt/azkaban/azkaban-exec/bin/start-exec.sh然后重新执行Job
参考资料：
Azkaban集群模式安装与execute-as-user配置：Azkaban集群模式安装与execute-as-user配置 - 简书
Azkaban编译安装配置文档：azkaban编译安装配置文档 - 丹江湖畔养蜂子赵大爹 - 博客园
这篇关于【Azkaban】 Missing required property ‘azkaban.native.lib‘ cause: null的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
