Programmer Think

1. Introduction to sqoop: Sqoop is an open source tool, mainly used in Hadoop(Hive) and traditional databases (mysql, postgresql ) Data can be transferred from one relational database (such as mysql, Oracle, Postgres, etc.) to another Data can be imported into HDFS of Hadoop or into relational database. ...

Article directory 4, DDL data definition 4.1 create database 4.2 query database 4.3 modify database 4.4 delete database 4.5 create table 4.6 zoning table 4.7 modification table 5, DML data operation 5.1 data import 5.2 data export 5.3 clear data in the table (Truncate) Six, query 6.1 basic query ...

Chapter 3 enterprise development cases 3.1 official case of monitoring port data Case requirements: first, Flume monitors port 44444 of the machine, then sends a message to port 44444 of the machine through telnet tool, and finally Flume displays the monitored data on the console in real time. Dema ...

Let's talk about the last step of preparing to upgrade presto, because the pre deployed Presto machine itself deployed a CDH cluster (customer's home), and the machine was Centos6.x. according to the JDK on the Cloudrea official website, it can't move. Later, it was found that his JDK environment variab ...

1 Definition Window function, also known as analysis function Analytic functions compute an aggregate value based on a group of rows. They differ from aggregate functions in that they return multiple rows for each group. The group of rows is called a window and is defined by the analytic_clause.Th ...

1.kerberos installation kerberos installation configuration and use: https://blog.csdn.net/qq_21383435/article/details/83625252 2. Generate keytab Execute the following command on cdh1 node, KDC server node: cd /var/kerberos/krb5kdc/ kadmin.local -q "addprinc -randkey hive/cdh-server1@YONG.COM " kadmin.local -q "addprinc - ...

1. There are many fields in the hive table, and one or several of them need to be removed from a large number of fields. It is not beautiful to write out all the fields, which can be completed by using regular Set hive.support.quoted.identifiers to None, and you can use regular expressions to get the fields of the table set h ...

1. hbase integration Mapreduce    in the offline task scenario, MapReduce accesses HBASE data to speed up analysis and expand analysis capabilities.Read data from hbase (result) public class ReadHBaseDataMR { private static final String ZK_KEY = "hbase.zookeeper.quorum"; private static final String ZK_VALUE = "hadoop01:2181,h ...

Hadoop 2.x.y (pseudo distributed) Refer to the single node setup section of the corresponding version of the official websitehttps://hadoop.apache.org/docs/ First, ssh and rsync Then download the bin package and extract it. Add the extracted root directory as the environment variable Hadoop? Home # example export HADOOP_HOME=/home/yzj/Applicat ...

Tuesday, 08. October 2019 11:01 am Initial preparation: 1. jdk installation Do the following on all three machines (depending on the number of machines you have): 1) you can install jdk through apt get, execute whereis java on the command line to get the installation path of java, or download the installation package of jdk manually f ...