本文主要是介绍Hadoop2.6.5单机安装，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
Hadoop2.6.5单机安装
JDK的安装
配置JDK环境变量
[root@spark1 soft]# vim /etc/profile
#JDK环境变量配置
#export JAVA_HOME=/application/jdk1.7.0_79
export JAVA_HOME=/application/jdk1.8.0_172
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib/rt.jar
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin环境变量生效
[root@spark1 soft]# source /etc/profile[root@spark1 soft]# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
[root@spark1 soft]#
配置SSH无密码登陆
$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys验证ssh，# ssh localhost
不需要输入密码即可登录。
Hadoop安装
下载
下载地址：
https://www.apache.org/dyn/closer.cgi/hadoop/common/
https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.6.5/hadoop-2.6.5.tar.gz
解压安装
[root@spark1 soft]# tar -zxvf hadoop-2.6.5.tar.gz -C /application/
创建hadoop安装所需目录
在/root /hadoop/目录下，建立tmp、hdfs/name、hdfs/data目录，执行如下命令
#mkdir /root/hadoop/tmp
#mkdir /root/hadoop/hdfs
#mkdir /root/hadoop/hdfs/data
#mkdir /root/hadoop/hdfs/name
设置Hadoop环境变量
#Hadoop环境变量配置
export HADOOP_HOME=/application/hadoop-2.6.5
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin[root@spark1 soft]# source /etc/profile
Hadoop配置
进入$HADOOP_HOME/etc/hadoop目录，配置 hadoop-env.sh等。涉及的配置文件如下：
1）hadoop-2.6.5/etc/hadoop/hadoop-env.sh
2）hadoop-2.6.5/etc/hadoop/yarn-env.sh
3）hadoop-2.6.5/etc/hadoop/core-site.xml
4）hadoop-2.6.5/etc/hadoop/hdfs-site.xml
5）hadoop-2.6.5/etc/hadoop/mapred-site.xml
6）hadoop-2.6.5/etc/hadoop/yarn-site.xml
1）配置hadoop-env.sh
# The java implementation to use.
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/application/jdk1.8.0_172
2）配置yarn-env.sh
# some Java parameters
# export JAVA_HOME=/home/y/libexec/jdk1.6.0/
export JAVA_HOME=/application/jdk1.8.0_172
3）配置core-site.xml
添加如下配置：
[root@spark1 hadoop]# cat core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration><property><name>fs.default.name</name><value>hdfs://spark1:9000</value><description>HDFS的URI，文件系统://namenode标识:端口号</description>
</property><property><name>hadoop.tmp.dir</name><value>/root/hadoop/tmp</value><description>namenode上本地的hadoop临时文件夹</description>
</property>
</configuration>[root@spark1 hadoop]#
4）配置hdfs-site.xml
[root@spark1 hadoop]# cat hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration>
<!--hdfs-site.xml-->
<property><name>dfs.name.dir</name><value>/root/hadoop/hdfs/name</value><description>namenode上存储hdfs名字空间元数据 </description>
</property><property><name>dfs.data.dir</name><value>/root/hadoop/hdfs/data</value><description>datanode上数据块的物理存储位置</description>
</property><property><name>dfs.replication</name><value>1</value><description>副本个数，配置默认是3,应小于datanode机器数量</description>
</property>
</configuration>
[root@spark1 hadoop]#
5）配置mapred-site.xml
[root@spark1 hadoop]# cat mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property><name>mapreduce.framework.name</name><value>yarn</value>
</property>
</configuration>
[root@spark1 hadoop]#
6）配置yarn-site.xml
[root@spark1 hadoop]# cat yarn-site.xml
<?xml version="1.0"?>
<configuration><!-- Site specific YARN configuration properties -->
<property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value>
</property>
<property><name>yarn.resourcemanager.webapp.address</name><value>${yarn.resourcemanager.hostname}:8099</value>
</property>
</configuration>
[root@spark1 hadoop]# 说明：
1）默认端口是8088；
2）这里我设置了yarn.resourcemanager.webapp.address为：${yarn.resourcemanager.hostname}:8099；
Hadoop启动
1）格式化namenode
hadoop namenode -format
2）启动NameNode 和 DataNode 守护进程
start-dfs.sh
3）启动ResourceManager 和 NodeManager 守护进程
start-yarn.sh
启动验证
1）执行jps命令，有如下进程，说明Hadoop正常启动
[root@spark1 soft]# jps
5649 DataNode
6631 ResourceManager
5815 SecondaryNameNode
5527 NameNode
6728 NodeManager
7981 Jps
[root@spark1 soft]#2）访问hdfs
http://192.168.2.191:50070
3）在浏览器中输入：http://192.168.2.191:8099/cluster 即可看到YARN的ResourceManager的界面。
注意：默认端口是8088，这里我设置了yarn.resourcemanager.webapp.address为：${yarn.resourcemanager.hostname}:8099
运行Hadoop的一个例子
[root@spark1 hadoop]# hadoop jar /application/hadoop-2.6.5/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar pi 2 100
Number of Maps = 2
Samples per Map = 100
19/04/13 13:46:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Wrote input for Map #0
Wrote input for Map #1
Starting Job
19/04/13 13:46:51 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
19/04/13 13:46:52 INFO input.FileInputFormat: Total input paths to process : 2
19/04/13 13:46:52 INFO mapreduce.JobSubmitter: number of splits:2
19/04/13 13:46:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1555134174372_0001
19/04/13 13:46:53 INFO impl.YarnClientImpl: Submitted application application_1555134174372_0001
19/04/13 13:46:53 INFO mapreduce.Job: The url to track the job: http://spark1:8099/proxy/application_1555134174372_0001/
19/04/13 13:46:53 INFO mapreduce.Job: Running job: job_1555134174372_0001
19/04/13 13:47:00 INFO mapreduce.Job: Job job_1555134174372_0001 running in uber mode : false
19/04/13 13:47:00 INFO mapreduce.Job: map 0% reduce 0%
19/04/13 13:47:14 INFO mapreduce.Job: map 100% reduce 0%
19/04/13 13:47:19 INFO mapreduce.Job: map 100% reduce 100%
19/04/13 13:47:19 INFO mapreduce.Job: Job job_1555134174372_0001 completed successfully
19/04/13 13:47:19 INFO mapreduce.Job: Counters: 49File System CountersFILE: Number of bytes read=50FILE: Number of bytes written=322803FILE: Number of read operations=0FILE: Number of large read operations=0FILE: Number of write operations=0HDFS: Number of bytes read=522HDFS: Number of bytes written=215HDFS: Number of read operations=11HDFS: Number of large read operations=0HDFS: Number of write operations=3Job Counters Launched map tasks=2Launched reduce tasks=1Data-local map tasks=2Total time spent by all maps in occupied slots (ms)=23209Total time spent by all reduces in occupied slots (ms)=2996Total time spent by all map tasks (ms)=23209Total time spent by all reduce tasks (ms)=2996Total vcore-milliseconds taken by all map tasks=23209Total vcore-milliseconds taken by all reduce tasks=2996Total megabyte-milliseconds taken by all map tasks=23766016Total megabyte-milliseconds taken by all reduce tasks=3067904Map-Reduce FrameworkMap input records=2Map output records=4Map output bytes=36Map output materialized bytes=56Input split bytes=286Combine input records=0Combine output records=0Reduce input groups=2Reduce shuffle bytes=56Reduce input records=4Reduce output records=0Spilled Records=8Shuffled Maps =2Failed Shuffles=0Merged Map outputs=2GC time elapsed (ms)=2514CPU time spent (ms)=12980Physical memory (bytes) snapshot=697511936Virtual memory (bytes) snapshot=6333603840Total committed heap usage (bytes)=499646464Shuffle ErrorsBAD_ID=0CONNECTION=0IO_ERROR=0WRONG_LENGTH=0WRONG_MAP=0WRONG_REDUCE=0File Input Format Counters Bytes Read=236File Output Format Counters Bytes Written=97
Job Finished in 28.254 seconds
Estimated value of Pi is 3.12000000000000000000
[root@spark1 hadoop]#
==============================
QQ群：143522604
群里有相关资源
欢迎和大家一起学习、交流、提升！
==============================
这篇关于Hadoop2.6.5单机安装的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
