Problem description
After configuring the Hadoop Namenode, secondarynode, and datanode nodes, configure mapred site in the hadoop /etc directory xml,yarn-site.xml file. After configuration, start resource manager and nodemanager. Among them, yen site The XML configuration is as follows:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>localhost.localdomain</value>
</property>
From the jps view, you can see that all started successfully. However, if you enter the following access address under windows, you will be prompted that the access timeout page cannot be found.
http://192.168.175.123:8088 Of which 192.168.175.123 Is the virtual machine address, 8088 is yarn Default port for management page
Problem solving
Check whether port 8088 is turned on. As shown below, you can see that port 8088 has been opened
The command to view a port process: lsof -i: port number, which can be used to check whether port 8088 is opened in JAVA.
[hadoop@localhost hadoop-2.8.5]$ lsof -i:8088 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java 3303 hadoop 234u IPv6 45723 0t0 TCP hadoop.fang.com:radan-http (LISTEN)
1. Windows browser access prompt connection timed out. The problem is that you need to turn off the firewall of the virtual machine. Different versions of linux have different shutdown methods. The Centos 7 shutdown method is as follows: set it to permanent shutdown. At this time, the windows browser will prompt connection rejected instead of connection timeout
- Temporarily turn off the firewall
systemctl stop firewalld - Permanent firewall on and off
systemctl disable firewalld - Temporarily open the firewall
systemctl start firewalld - Firewall startup
systemctl enable firewalld - View firewall status
systemctl status firewalld
2.netstat -nltp view the port enabling status, as shown in the following figure, You can see port 8088 (yarn interface) and port 50070 (Hadoop interface) is mounted under the IP of 127.0.0.1. Therefore, this port can only be accessed locally. So the problem lies here. You need to mount the port under the IP of the virtual machine. First, modify the hostname parameter configuration of yarn-site.xml to one of your own domain names, or write it directly as the IP of the virtual machine. My configuration is hadoop.fang.com. If it is configured as a domain name If so, you need to add domain name resolution to / etc/hosts. The operation is as follows
[root@venn05 hadoop]# netstat -nltp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:35115 0.0.0.0:* LISTEN 9344/java tcp 0 0 0.0.0.0:9868 0.0.0.0:* LISTEN 8816/java tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 127.0.0.1:8020 0.0.0.0:* LISTEN 8467/java tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1203/dnsmasq tcp 0 0 127.0.0.1:50070 0.0.0.0:* LISTEN 8467/java tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 972/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 971/cupsd tcp 0 0 127.0.0.1:8088 0.0.0.0:* LISTEN 9048/java tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1086/master tcp 0 0 0.0.0.0:13562 0.0.0.0:* LISTEN 9344/java tcp 0 0 127.0.0.1:8030 0.0.0.0:* LISTEN 9048/java tcp 0 0 127.0.0.1:8031 0.0.0.0:* LISTEN 9048/java tcp 0 0 127.0.0.1:8032 0.0.0.0:* LISTEN 9048/java tcp 0 0 127.0.0.1:8033 0.0.0.0:* LISTEN 9048/java tcp 0 0 0.0.0.0:8040 0.0.0.0:* LISTEN 9344/java tcp 0 0 0.0.0.0:8042 0.0.0.0:* LISTEN 9344/java tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 972/sshd tcp6 0 0 ::1:631 :::* LISTEN 971/cupsd tcp6 0 0 ::1:25 :::* LISTEN 1086/master
Note: the netstat command is used to print the status information of the network system in Linux, which can let you know the network situation of the whole Linux system.
-n or--numeric: Direct use ip Address, not through the domain name server; -l or--listening: Displays the name of the server under monitoring Socket； -t or--tcp: display TCP Connection status of transmission protocol; -p or--programs: Show in use Socket Program identification code and program name;
yarn-site. The XML configuration is as follows:
<configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.hostname</name> <value>hadoop.fang.com</value> </property> </configuration>
/The configuration of etc/hosts is as follows. The domain name must be configured as the local IP address of the virtual machine.
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 192.168.175.123 hadoop.fang.com
Note: curl command is a file transfer tool that uses URL rules to work on the command line. It supports file upload and download, so it is a comprehensive transmission tool, but traditionally, curl is called a download tool. Through - I or - head, you can only print the HTTP header information: the command curl -I http address
[hadoop@localhost hadoop-2.8.5]$ curl -I http://192.168.175.123:50070 HTTP/1.1 200 OK Cache-Control: no-cache Expires: Thu, 27 Sep 2018 05:49:48 GMT Date: Thu, 27 Sep 2018 05:49:48 GMT Pragma: no-cache Expires: Thu, 27 Sep 2018 05:49:48 GMT Date: Thu, 27 Sep 2018 05:49:48 GMT Pragma: no-cache Content-Type: text/html; charset=utf-8 X-FRAME-OPTIONS: SAMEORIGIN Expires: Thu, 27 Sep 2018 05:49:48 GMT Date: Thu, 27 Sep 2018 05:49:48 GMT Pragma: no-cache Expires: Thu, 27 Sep 2018 05:49:48 GMT Date: Thu, 27 Sep 2018 05:49:48 GMT Pragma: no-cache X-FRAME-OPTIONS: SAMEORIGIN Last-Modified: Mon, 10 Sep 2018 03:58:15 GMT Content-Length: 1079 Accept-Ranges: bytes
reference resources: https://www.codeleading.com/article/3408122164/