本文尝试在CentOS上搭建10台HDFS集群,不会搭建YARN和Hive, 因为之后会采用Spark。先安装jdk 1.8, 此处不细说。
首先下载hadoop binary包
wget http://apache.mesi.com.ar/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz
创建管理hadoop集群的帐号
useradd lisa
可以用批处理的方式对10台服务器都创建lisa帐号,主要通过ssh -t 这里不再细说
修改目录拥有者
chown -R lisa:lisa hadoop-2.7.1/ chown -R lisa:lisa hadoop ll total 8 drwxr-xr-x 3 root root 4096 Sep 25 18:02 download lrwxrwxrwx 1 lisa lisa 13 Sep 25 17:42 hadoop -> hadoop-2.7.1/ drwxr-xr-x 9 lisa lisa 139 Jun 29 14:15 hadoop-2.7.1 lrwxrwxrwx 1 root root 24 Sep 25 16:50 java -> /data/slot0/jdk1.8.0_45/ drwxr-xr-x 8 root root 4096 Sep 25 14:01 jdk1.8.0_45
切换到hadoop系统使用的帐号,然后生成ssh key
su - lisa ssh-keygen -t rsa -P '' Generating public/private rsa key pair. Enter file in which to save the key (/home/lisa/.ssh/id_rsa): Created directory '/home/lisa/.ssh'. Your identification has been saved in /home/lisa/.ssh/id_rsa. Your public key has been saved in /home/lisa/.ssh/id_rsa.pub. The key fingerprint is: 15:44:c6:4e:bb:cd:b9:09:e7:50:c7:ae:dd:92:56:57 lisa@10-149-11-152 The key's randomart image is: +--[ RSA 2048]----+ | += | | .o. | | o.. . | | .o . o E| | S = + .| | + = . o| | = = +.| | = = .| | . . | +-----------------+