本文主要是介绍flume加载问题整理,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
failed to start agent because dependencies were not found in classpath
flume-env.sh中需要同时设置JAVA_HOME和HCAT_HOME
failed to connecting to Endpoint
hive服务器没有启动bin/hive --service metastore &bin/hive --service hiveserver2 &
transaction property is not set on table
表transactions未设置,创建表时需要添加CREATE TABLE students5 (name VARCHAR(64), age INT, gpa DECIMAL(3, 2)) CLUSTERED BY (age) INTO 2 BUCKETS STORED AS ORC TBLPROPERTIES("transactional"="true");
transactionBatch txnIds=XXX on endPoint=XXX has been closed
分隔符设置问题:前一个双引号,后一个单引号agent1.sinks.alarmk2.serializer.delimiter=", " agent1.sinks.alarmk2.serializer.serdeSeparator =","
改成:
agent1.sinks.alarmk2.serializer.delimiter="\t" agent1.sinks.alarmk2.serializer.serdeSeparator ='\t'
This command is not allowed on an ACID table XXX with a non-ACID
transaction manager
问题出在:当设置成TBLPROPERTIES(“transactional”=”true”),hive中需要在配置文件中设置:Client Side
- hive.support.concurrency – true
- hive.enforce.bucketing – true (Not required as of Hive 2.0)
- hive.exec.dynamic.partition.mode – nonstrict
- hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager
Server Side (Metastore)
- hive.compactor.initiator.on – true (See table below for more details)
- hive.compactor.worker.threads – a positive number on at least one instance of the Thrift metastore service
官网中有介绍:https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions
查看hive中数据出现”NULL“,行数据均作为一个属性填入数据库,其它属性默认为空
设置数据属性间用”\t“分割
The channel is full, and cannot write data now
内存满
将channel的参数capacity、transactionCapacity调大
这篇关于flume加载问题整理的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!