本文主要是介绍问题报错org.apache.hadoop.fs.ChecksumException: Checksum error: file:/F:/part-r-00000(已解决),希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
文章目录
- 问题样例
- 简单粗暴的解决办法
问题样例
org.apache.hadoop.fs.ChecksumException: Checksum error: file:/F:/design/userOfResult/part-r-00000 at 0 exp: -1229056335 got: 168623484at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:323)at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:279)at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:228)at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:196)at java.io.DataInputStream.read(DataInputStream.java:100)at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)at org.apache.hadoop.mapred.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:206)at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:244)at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47)at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:266)at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:211)at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.agg_doAggregateWithoutKey$(Unknown Source)at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown Source)...
简单粗暴的解决办法
分析出错日志,可以断定是无法检测到文件来源导致报错,需要将待分析的文件,换一个路径,重新运行就ok了。
这篇关于问题报错org.apache.hadoop.fs.ChecksumException: Checksum error: file:/F:/part-r-00000(已解决)的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!