sqoop data export and import

1. Introduction to sqoop: Sqoop is an open source tool, mainly used in Hadoop(Hive) and traditional databases (mysql, postgresql ) Data can be transferred from one relational database (such as mysql, Oracle, Postgres, etc.) to another Data can be imported into HDFS of Hadoop or into relational database. ...

Posted by wolfrock on Wed, 26 Feb 2020 07:30:01 +0100

Basic operation of Hive

Article directory 4, DDL data definition 4.1 create database 4.2 query database 4.3 modify database 4.4 delete database 4.5 create table 4.6 zoning table 4.7 modification table 5, DML data operation 5.1 data import 5.2 data export 5.3 clear data in the table (Truncate) Six, query 6.1 basic query ...

Posted by PyroX on Sun, 23 Feb 2020 09:38:46 +0100

Quick learning - Flume enterprise development case

Chapter 3 enterprise development cases 3.1 official case of monitoring port data Case requirements: first, Flume monitors port 44444 of the machine, then sends a message to port 44444 of the machine through telnet tool, and finally Flume displays the monitored data on the console in real time. Dema ...

Posted by poring on Thu, 13 Feb 2020 21:26:22 +0100

Presto cluster deployment (upgrade)

Let's talk about the last step of preparing to upgrade presto, because the pre deployed Presto machine itself deployed a CDH cluster (customer's home), and the machine was Centos6.x. according to the JDK on the Cloudrea official website, it can't move. Later, it was found that his JDK environment variab ...

Posted by B of W on Tue, 11 Feb 2020 11:48:16 +0100

Making clear of hive window function

1 Definition Window function, also known as analysis function Analytic functions compute an aggregate value based on a group of rows. They differ from aggregate functions in that they return multiple rows for each group. The group of rows is called a window and is defined by the analytic_clause.Th ...

Posted by xsgatour on Thu, 06 Feb 2020 15:06:54 +0100

hive opens the Kerberos beeline connection

1.kerberos installation kerberos installation configuration and use: https://blog.csdn.net/qq_21383435/article/details/83625252 2. Generate keytab Execute the following command on cdh1 node, KDC server node: cd /var/kerberos/krb5kdc/ kadmin.local -q "addprinc -randkey hive/cdh-server1@YONG.COM " kadmin.local -q "addprinc - ...

Posted by smpdawg on Tue, 10 Dec 2019 17:22:35 +0100

hive operation summary

1. There are many fields in the hive table, and one or several of them need to be removed from a large number of fields. It is not beautiful to write out all the fields, which can be completed by using regular Set hive.support.quoted.identifiers to None, and you can use regular expressions to get the fields of the table set h ...

Posted by codects on Mon, 09 Dec 2019 18:16:24 +0100

Typical scenes of hbase

1. hbase integration Mapreduce    in the offline task scenario, MapReduce accesses HBASE data to speed up analysis and expand analysis capabilities.Read data from hbase (result) public class ReadHBaseDataMR { private static final String ZK_KEY = "hbase.zookeeper.quorum"; private static final String ZK_VALUE = "hadoop01:2181,h ...

Posted by diagnostix on Wed, 04 Dec 2019 11:55:27 +0100

Archlinux/Manjaro install MariaDB Hadoop Hive (pseudo distributed)

Hadoop 2.x.y (pseudo distributed) Refer to the single node setup section of the corresponding version of the official websitehttps://hadoop.apache.org/docs/ First, ssh and rsync Then download the bin package and extract it. Add the extracted root directory as the environment variable Hadoop? Home # example export HADOOP_HOME=/home/yzj/Applicat ...

Posted by beerman on Sat, 30 Nov 2019 22:12:43 +0100

Cluster construction of hadoop, spark, hive and azkaban under ubuntu

Tuesday, 08. October 2019 11:01 am Initial preparation: 1. jdk installation Do the following on all three machines (depending on the number of machines you have): 1) you can install jdk through apt get, execute whereis java on the command line to get the installation path of java, or download the installation package of jdk manually f ...

Posted by mattal999 on Sat, 02 Nov 2019 11:47:52 +0100