HDFS high availability architecture

Posted by antileon on Sat, 26 Feb 2022 14:10:24 +0100

First, we need to build three virtual machines (here is a demonstration). For the construction process, please refer to this article.

Using virtual machine to complete Hadoop fully distributed construction_ You can read your own blog - CSDN blog

After completing the Hadoop fully distributed construction in the previous article, you can do the next operation - build a Zookeeper cluster

First download Zookeeper for Linux

Link: https://pan.baidu.com/s/1_alxqyJMAiCkORtM37RT0Q  
Extraction code: hjl4

After downloading, use the command to decompress

Upload the file to / usr/local (the upload method is demonstrated in the above file)

tar -zxvf apache-zookeeper-3.7.0-bin.tar -C ~/home/hadoop/

After decompression, enter the conf directory of zookeeper, and set zoo sampli. The cfg file is modified to zoo cfg

cp zoo.sample.cfg zoo.cfg

Modify zoo cfg

vi cfg

Modify dataDir to a custom directory


Add the following at the end of the document


1, 2 and 3 in the above configuration refer to the number of each node of the zookeeper cluster. The contents separated by colons on the right side of the equal sign refer to the corresponding host name, heartbeat port between nodes and data interaction port respectively

zoo. After CFG modification, create the dataDir directory specified therein, and save the corresponding node number in the configuration to a file named myid

mkdir  /home/hadoop/zkdata

echo 1 >/home/hadoop/zkdata/myid

Distribute the pressurized zookeeper folder to slave1 and slave2

scp -r apache-zookeeper-3.7.0-bin root@slave1:/home/hadoop/

scp -r /home/hadoop/apache-zookeeper-3.7.0-bin root@slave2:/home/hadoop

Create zkdata on slave1 and slave2 respectively

mkdir  /home/hadoop/zkdata

echo 2 >/home/hadoop/zkdata/myid
mkdir  /home/hadoop/zkdata

echo 3 >/home/hadoop/zkdata/myid

Add zookeeper environment variable

vi /etc/profile

export PATH=$PATH:/home/hadoop/apache-zookeeper-3.7.0-bin/bin

Start zookeeper and check whether the installation is successful

zkServer.sh start

zkServer.sh status



Seeing this, it shows that zookeeper has been successfully installed

zookeeper has been successfully installed. Next, set up the HDFS HA cluster

The HDFS cluster has been installed in the above article. Now directly modify the configuration file to achieve the HA configuration.

First, modify the core site XML file

vi /usr/local/hadoop/etc/hadoop/core-site.xml

Add the following content under the configuration file

 <!-- appoint hdfs of nameservice by ns -->
                <!-- HDFS Resource path -->

        <!-- appoint ZooKeeper address -->

Modify HDFS site XML file

vi /usr/local/hadoop/etc/hadoop/hdfs-site.xml

Add the following

        <!-- appoint hdfs of nameservice by ns,Need and core-site.xml Consistent in -->
        <!-- ns There are two below NameNode,part nn1,nn2 -->
        <!-- nn1 of RPC mailing address -->
        <!-- nn1 of http mailing address -->
        <!-- nn2 of RPC mailing address -->
        <!-- nn2 of http mailing address -->
        <!-- appoint NameNodeN Metadata for JournalNode Storage location on -->
        <!-- appoint Journal Node Location of data on local disk -->
        <!-- open NameNode Automatic switching in case of failure -->
        <!-- Implementation mode of automatic switching in case of configuration failure -->
                <name>dfs.client.failover.proxy.provider.ns</name>             <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
        <!-- Configure isolation mechanism -->
        <!-- Required when using isolation mechanism SSH Login free -->

Modify mapred site xml

vi /usr/local/hadoop/etc/hadoop/mapred-site.xml

Add the following


Modify Yard site XML file

vi /usr/local/hadoop/etc/hadoop/yarn-site.xml

Add the following

 <!-- appoint nodemanajager Load on startup server The method of is shuffle server -->
  <!-- appoint resourcemanager address -->

Next, modify the slave file (some are workers files due to different versions, and Hadoop version 3. X or above)

Add the following


Distribute the configured hadoop to slave1 and slave2

scp -r /usr/local/hadoop/etc/hadoop root@slave1:/usr/local/hadoop/etc/

scp -r /usr/local/hadoop/etc/hadoop root@slave2:/usr/local/hadoop/etc/

Next is the various services required by the HA cluster

zkServer. SH start

Start JournalNode

hadoop-daemon.sh start journalnode

After execution, use jps to view the process

The process in the box needs to be included, which means that the startup is successful

Then format HDFS

Execute the command in the master: hdfs namenode -format

After successful execution, it will be displayed on the core site Hadoop. XML configuration tmp. Create the corresponding data directory under dir directory. The structure is as follows:

Tree TMP (if the command doesn't work, execute Yum - y install tree first)

Since slave1 also needs to start NameNode, copy the generated directory to the directory corresponding to slave1

Format ZKFC

hdfs zkfc -formatZK

After formatting successfully, a large number of logs will be output


Then start HDFS and YARN clusters

Execute start DFS sh/start-yarn. SH command

Using jps to view processes


Ignore the process in the red box

On the master node, enter hdfs haadmin -getAllServiceState

I've switched here. According to the above, it should be master to act

This is accessible http://master:9870 and http://slave1:9870


Through jps, you can see that the process number of NameNode in slave1 is 57946. Use kill -9 57946to kill the process. Check whether the NameNode can be switched normally. Enter address: http://slave1:9870  


So far, the HDFS high availability configuration has been demonstrated. The next article is about the YARN high availability configuration.

Topics: Big Data Hadoop hdfs