Centos6 installation CDH5.15.1 most detailed Version-1

Posted by delphi123 on Sun, 22 Dec 2019 22:57:07 +0100

Here are the three virtual machines installed before, as follows:
192.168.201.81 hadoop-1
192.168.201.82 hadoop-2
192.168.201.83 hadoop-3

preparation:

  1. Configure static IP
vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:0C:29:AF:65:27
TYPE=Ethernet
UUID=269569e1-f888-4f73-b2f7-7af45225b572
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=none
IPADDR=192.168.201.81
NETMASK=255.255.255.0
GATEWAY=192.168.201.1

Others remain unchanged. The following configuration items are mainly configured:
ONBOOT:IP boot automatically
BOOTPROTO:IP type is static IP
IPADDR:IP address
NETMASK: subnet mask
GATEWAY: default GATEWAY
DNS1,DNS2:DNS server (for networking)

Restart the network card driver

service network restart

All three of them should be configured.

  1. Modify host name

My host names are respectively hadoop-1, hadoop-2 and hadoop-3

  1. hosts file host name and IP address mapping
    In addition, you'd better add the following Windows path:
    C:\Windows\System32\drivers\etc\hosts
vim /etc/hosts

## Add the following
192.168.201.81 hadoop-1
192.168.201.82 hadoop-2
192.168.201.83 hadoop-3
  1. configure network
    Modify / etc/resolv.conf
vi /etc/resolv.conf
#Add the following
nameserver 8.8.8.8
  1. Install common tools

My system is to minimize the installation, many components do not have to be installed by themselves, such as wget,vim,lrzsz, etc.

yum -y install wget
yum -y install vim
yum -y install vim lrzsz
  1. Disable IPV6
    The command line executes the following command
## Disable IPv6
echo "                   " >> /etc/modprobe.d/dist.conf
echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf
echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf
## Check whether the appending is successful
tail /etc/modprobe.d/dist.conf

  1. Turn off firewall
# Turn off firewall
service iptables stop
# Turn off firewall permanently
chkconfig iptables off
# View firewall status:
service iptables status
  1. Disable Selinux
    Restart the machine will take effect. Do not restart here
vim /etc/sysconfig/selinux
## To modify a configuration item:
SELINUX=disabled

  1. Unload the JDK provided by the system
    rpm -qa|grep java
# xxx yyy zzz is the plug-in you want to uninstall. The plug-ins are separated by spaces
rpm -e --nodeps xxx yyy zzz


I have nothing here.

  1. Set the number of file opens and the maximum number of user processes
## Set the maximum number of user processes
vim /etc/security/limits.conf
## Add the following at the end
*       soft    nofile          32768
*       hard    nofile          1048576
*       soft    nproc           65536
*       hard    nproc           unlimited
*       soft    memlock         unlimited
*       hard    memlock         unlimited

  1. The unified time zone is Zone 8
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
  1. Linux operating system language adopts English version
# View operating system language
echo $LANG
# Modify operating system language
vim /etc/sysconfig/i18n
LANG="en_US.UTF-8"
  1. Close large transparent page
# Temporarily close transparent large page (effective immediately)
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
# Permanently close the transparent large page (restart takes effect)
echo '            ' >> /etc/rc.local
echo '# Close the big transparent page '> > / etc / rc.local
echo 'echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled' >> /etc/rc.local
# See if it's closed
cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
  1. Modify swappiness
# Temporary modification
sysctl -w vm.swappiness=1
# Permanent modification
echo "vm.swappiness=1" >> /etc/sysctl.conf
  1. Restart the server
reboot

Topics: Hadoop vim network firewall