Programmer Think

Create a table with the NAME hbase_. The HBase table is composed of Key values. In this table, the Key is NAME   This table has two column families, CF1 and CF2. Under CF1 and CF2, there are two columns, name and gender, Chinese and Math 1. There are two column families CF1 and CF2 in creating table HBase hbase(main):041:0&g ...

The Spark mentioned here includes SparkCore/SparkSQL/SparkStreaming. In fact, all operations are the same. The following shows the code in the actual project. Method 1: write the entire DataFrame to MySQL at one time (the Schema of DataFrame should be consistent with the domain name defined in the MySQL table) Dat ...

1. Demand scenario Migrate part of the data in the ys table in HBase to the ys? Mr table through Mapreduce 2. Coding 1) build ReadysMapreduce class to read data in ys table package cn.ysjh; import java.io.IOException; import org.apache.hadoop.hbase.Cell; import org.apache.hadoop.hbase.CellUtil; import org.apache.hadoop.hb ...

Python crawls the information of the broker Python2.7.15 Today, we're going to crawl through the information of the settlement agent. This time we don't use regular, we use beautiful soup. If you don't know about it, you can read this document first for easy understanding. https://beautifulsoup.readthedocs.io/zh_CN/v4.4.0/ I ...

1.kerberos installation kerberos installation configuration and use: https://blog.csdn.net/qq_21383435/article/details/83625252 2. Generate keytab Execute the following command on cdh1 node, KDC server node: cd /var/kerberos/krb5kdc/ kadmin.local -q "addprinc -randkey hive/cdh-server1@YONG.COM " kadmin.local -q "addprinc - ...

prerequisite: 1. Install jdk1.8 (under Windows Environment) 2. Install maven 3.3.9 (under Windows Environment) 3. Install eclipse (under Windows Environment) 4. Install hadoop (under Linux environment) Question: The input file is: EMP.csv. The contents of the EMP.csv file are as follows: SAL is employee salary (int type), ...

I. flume installation 1. decompress tar -zxvf flume-ng-1.6.0-cdh5.15.0.tar.gz -C /opt/modules/ 2. Change name mv apache-flume-1.6.0-cdh5.15.0-bin/ flume-1.6.0-cdh5.15.0-bin/ 3. Configuration file: flume-env.sh export JAVA_HOME=/opt/modules/jdk1.8.0_151 4. Test success bin/flume-ng version //Result: Flume 1.6.0-cdh5. ...

1. There are many fields in the hive table, and one or several of them need to be removed from a large number of fields. It is not beautiful to write out all the fields, which can be completed by using regular Set hive.support.quoted.identifiers to None, and you can use regular expressions to get the fields of the table set h ...

[Hadoop cluster building] ssh password free login setting There are three hosts, their IP addresses and roles in the cluster are as follows: 172.17.0.2 //master 172.17.0.3 //slave1 172.17.0.4 //slave2 Now you want to make the two of them able to log in through ssh, the steps are as follows: Create public key and configure acc ...

1. Problem description Use Impala JDBC to insert Chinese characters into Kudu table. The inserted Chinese characters are garbled and the Chinese characters are truncated. After the previous document uses sql splicing method to insert Chinese string scrambling solution, this document describes the problem of using jdbc PreparedStatement method t ...