Programmer Think

1. Introduction to zookeeper Zookeeper is a distributed service framework, which used to be a sub project of Apache Hadoop. Now it is an independent top-level project of Apache. It is mainly used to solve some data management problems often encountered in distributed applications, such as unified nami ...

Configuration environment: Mac OS 10.14.5 hadoop version: 3.2.1 Time: February 29, 2020 Install Homebrew Homebrew is commonly used on mac, with few descriptions and installation methods /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" SSH login loca ...

1. Introduction to sqoop: Sqoop is an open source tool, mainly used in Hadoop(Hive) and traditional databases (mysql, postgresql ) Data can be transferred from one relational database (such as mysql, Oracle, Postgres, etc.) to another Data can be imported into HDFS of Hadoop or into relational database. ...

Article directory 4, DDL data definition 4.1 create database 4.2 query database 4.3 modify database 4.4 delete database 4.5 create table 4.6 zoning table 4.7 modification table 5, DML data operation 5.1 data import 5.2 data export 5.3 clear data in the table (Truncate) Six, query 6.1 basic query ...

Chapter 3 enterprise development cases 3.1 official case of monitoring port data Case requirements: first, Flume monitors port 44444 of the machine, then sends a message to port 44444 of the machine through telnet tool, and finally Flume displays the monitored data on the console in real time. Dema ...

** Install Centos6 **1, Download the CentOS 6.10 installation package, link . Install in VMWare. 2, Set virtual machine fixed ip Edit - > virtual network editor - > device NAT mode - > View gateway. Right click virtual machine - > set NAT mode 2. Modify the network card configuration. ...

hadoop installation and configuration First, decompression. 2, Modify Hadoop configuration file 4. Configure environment variables 5. Copy the installation package 6. Start cluster 7. Cluster status 8. Monitoring page 9. Shut down the cluster First, decompression. tar -zxvf hadoop-2.8.2.tar.gz 2, Modify Hadoop confi ...

The partitioner of MapReduce is HashPartitionerPrinciple: first, hash the key output from the map, then reduce the number of tasks on the module. According to the result, determine the output kv pair, which is taken by the matching reduce task.Custom partition needs to inherit the Partitioner and copy the getpariton() methodCustom partition cl ...

Here are the three virtual machines installed before, as follows: 192.168.201.81 hadoop-1 192.168.201.82 hadoop-2 192.168.201.83 hadoop-3 preparation: Configure static IP vi /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 HWADDR=00:0C:29:AF:65:27 TYPE=Ethernet UUID=269569e1-f888-4f73-b2f7-7af45225b572 ONBOOT=yes NM_CONT ...

1. Demand scenario Migrate part of the data in the ys table in HBase to the ys? Mr table through Mapreduce 2. Coding 1) build ReadysMapreduce class to read data in ys table package cn.ysjh; import java.io.IOException; import org.apache.hadoop.hbase.Cell; import org.apache.hadoop.hbase.CellUtil; import org.apache.hadoop.hb ...