How to connect hbase stand-alone version (using external zk) and java api

0 create modify information

time	content	remarks
20210927	create documents

0 version

Component name	edition	Download address
operating system	centos7	CentOS-7-x86_64-DVD-2009.iso
jdk	1.8	jdk-8u301-linux-x64.tar.gz
hadoop	3.0.0	hadoop-3.0.0.tar.gz
zookeeper	3.4.5	zookeeper-3.4.5.tar.gz
hbase	2.0.0	hbase-2.0.0-bin.tar.gz

0 attention

The hosts file should configure the public ip and private ip to correspond to the same host name. Try to use the host name instead of ip in the configuration file

The local code development configuration file uses the public ip + port, and the Intranet can use the public ip or the intranet ip + port

(the intranet ip can access the public ip. If not, check whether the firewall is turned off and whether the Alibaba cloud server security group develops ports.)

No secret free operation is required for single hbase installation

Pseudo distributed requires hadoop and zk to be installed, and secret free operation is required

1. Configure environment variables

Modify profile

vi /etc/profile

Add the following

Add / opt/myinstall/jdk1.8.0_144 change this path to the path where your installation package is located

#java
export JAVA_HOME=/opt/myinstall/jdk1.8.0_144
export PATH=$PATH:$JAVA_HOME/bin
#zookeeper
export ZOOKEEPER_HOME=/opt/myinstall/hbase/zookeeper-3.4.5
export PATH=$PATH:$ZOOKEEPER_HOME/bin
#hbase
export HBASE_HOME=/opt/myinstall/hbase/hbase-2.0.0
export PATH=$PATH:$HBASE_HOME/bin

Make the configuration just effective

source /etc/profile

Modify hosts file

 vi /etc/hosts

Configure public network ip Private network ip Corresponding host name iZbp1xxx85opgretwdxZ 
101.xx.xxx.xx   iZbp1xxx85opgretwdxZ 
172.xx.xx.xx    iZbp1xxx85opgretwdxZ 

2. Install jdk

Enter the above environment variables after they are configured

java -version

If the following contents appear, the installation is successful

[root@template-v3 ~]# java -version
java version "1.8.0_301"
Java(TM) SE Runtime Environment (build 1.8.0_301-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode)

3 install zookeeper

Enter the installation directory cd /opt/myinstall/hbase/zookeeper-3.4.5
Create data directory mkdir data
Create log directory mkdir log
Enter the directory cd /opt/myinstall/hbase/zookeeper-3.4.5/conf
Copy the configuration file and rename it cp zoo_sample.cfg zoo.cfg

Modify the configuration file vi zoo.cfg

Modify the data directory and add the log directory to your own actual path. Other configurations do not need to be changed

dataDir=/opt/myinstall/hbase/zookeeper-3.4.5/data
dataLogDir=/opt/myinstall/hbase/zookeeper-3.4.5/log

Start zookeeper

start-up zk  zkServer.sh start
 stop it zk   zkServer.sh stop
 see zk state  zkServer.sh status

View process jps

jps

30741 Jps
27592 QuorumPeerMain

4. Install hbase

4.1 modifying configuration files

Create the path to be configured in the configuration file

cd /opt/myinstall/hbase/hbase-2.0.0/
mkdir data
mkdir zkdata

Enter the directory where the hbase configuration file is located

cd /opt/myinstall/hbase/hbase-2.0.0/conf

Modify hbase-env.sh file

vi hbase-env.sh

Add java environment variables and settings without using internal zk

\# The java implementation to use.  Java 1.8+ required.

export JAVA_HOME=/opt/myinstall/jdk1.8.0_144/

 

\# Tell HBase whether it should manage it's own instance of ZooKeeper or not.

export HBASE_MANAGES_ZK=false

Modify the hbase-site.xml file, add the following contents, and change the relevant path to your own path

vi hbase-site.xml 

<configuration>
 <!-- hbase Storage data directory -->
  <property>
    <name>hbase.rootdir</name>
    <value>file:opt/myinstall/hbase/hbase-2.0.0/data</value>
  </property>

  <!-- ZooKeeper Data file path -->
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/opt/myinstall/hbase/hbase-2.0.0/zkdata</value>
  </property>

<!--appoint zk The address is the machine name-->
<property>
        <name>hbase.zookeeper.quorum</name>
        <value>iZbp1xxx85opgretwdxZ</value>
</property>
<property>
  <name>hbase.zookeeper.property.clientPort</name>
  <value>2181</value>
</property>
 <!-- ZooKeeper storage hbase Node name of the data -->
    <property>
        <name>zookeeper.znode.parent</name>
        <value>/hbase</value>
    </property>
    <!-- Cluster mode, distributed mode or stand-alone mode, if set to false If so, HBase Process and Zookeeper The process is in the same JVM Process stand-alone setup true Will report an error -->
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
<property>
  <name>hbase.unsafe.stream.capability.enforce</name>
  <value>false</value>
</property>
</configuration>

4.2 start hbase

start-up hbase   start-hbase.sh
 stop it hbase   stop-hbase.sh
 implement hbase crud command  hbase shell

View process jps

jps

30741 Jps

27592 QuorumPeerMain

28715 HRegionServer

30060 HMaster

Check for success

input hbase shell
 Execute a command casually, such as list_namespace View namespace
 If it can be displayed normally, it indicates success as follows

hbase(main):016:0> list_namespace
NAMESPACE                                                                                                    default                                                                                                     hbase                                                                                                       2 row(s)
Took 0.0461 seconds

4.3 viewing logs

cd $HBASE_HOME/logs
cat hbase-root-master-iZbp1xxx85opgretwdxZ .log

4.4 Hbase startup error

Error: java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use. hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal. dir' points to a FileSystem mount that can provide it.

hbase-site.xml Add the following content to, which has been added above
<property>
  <name>hbase.unsafe.stream.capability.enforce</name>
  <value>false</value>
</property>

5 how to connect hbase with Java API in the simplest way

5.1 code

Dependency of pom file

<dependencies>
    <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-server -->
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-server</artifactId>
        <version>2.0.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-client -->
    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-client</artifactId>
        <version>2.0.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper -->
    <dependency>
        <groupId>org.apache.zookeeper</groupId>
        <artifactId>zookeeper</artifactId>
        <version>3.4.6</version>
        <type>pom</type>
    </dependency>
</dependencies>

The code creates a namespace to test whether the connection is successful

package com.xhxtest;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.NamespaceDescriptor;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.log4j.Logger;

import java.io.IOException;

/**
 * @Author xiehongxin
 * @Date 2021/7/30
 * @Description //TODO
 */
public class Demo {

    /**
     * @return void
     * @Author xiehongxin
     * @Description //TODO  test
     * @Date 18:29 2021/9/17
     * @Param args
     **/
    public static void main(String[] args) throws IOException {
        System.out.println("start");
        System.out.println(Demo.createNameSpace("xns")); //Create namespace
        System.out.println("end");

    }


    private static Logger logger = Logger.getLogger(Demo.class);

    public static Configuration configuration; // Manage configuration information for Hbase
    public static Connection connection; // Manage Hbase connections
    public static Admin admin; // Manage information for Hbase database

    // Establish a connection before operating the database
    static {
        //The local configuration package should be removed
        System.setProperty("hadoop.home.dir", "D:\\XHXFile\\apache\\hadoop-2.7.3-common-winutils");
        //Core configuration
        configuration = HBaseConfiguration.create();

        configuration.set("hbase.zookeeper.quorum", "101.xx.xxx.xx"); // Set the public ip address of the zookeeper node
        configuration.set("hbase.zookeeper.property.clientPort", "2181"); // Set zookeeper node port
        try {
            connection = ConnectionFactory.createConnection(configuration);
            admin = connection.getAdmin();
        } catch (IOException e) {
            e.printStackTrace();
            logger.error("====Hbase initialization failed====" + e);
        }

    }

    /**
     * @return java.lang.Boolean
     * @Author xiehongxin
     * @Description //TODO  Create namespace
     * @Date 17:25 2021/9/17
     * @param: namespaceName
     **/
    public static Boolean createNameSpace(String namespaceName) {
        if (admin == null) {
            throw new RuntimeException("===please check admin is init?===");
        }
        Boolean result = false;
        NamespaceDescriptor mkNameSpace = NamespaceDescriptor.create(namespaceName).build();
        try {
            admin.createNamespace(mkNameSpace);
            result = true;
        } catch (IOException e) {
            e.printStackTrace();
        }
        return result;
    }


}

5.2 java api error

java api Connection error: Call exception, tries=10, retries=31, started=54309 ms ago, cancelled=false, msg=

Modify the hosts file of the local computer C:\Windows\System32\drivers\etc\hosts

Add the same content as the linux hosts file

101.xx.xxx.xx   iZbp1xxx85opgretwdxZ 
172.xx.xx.xx    iZbp1xxx85opgretwdxZ