Programmer Think

Design pattern in java: template pattern Define the skeleton (which is abstracted by the general algorithm) and hand over the specific implementation to the subclass. This means that as long as the process is defined in the template and how to implement it, the template method does not pay attention to the specific implementation. The specific ...

HDFS transparent encryption, Keystore and Hadoop KMS, encryption area The data in HDFS will be saved in the form of blocks in the local disk of each data node, but these blocks are in clear text. If you directly access the directory where the block is located under the operating system, you can directly view the contents through the cat comman ...

Absrtact: This article mainly introduces the use of the basic api of hadoop hdfs. Including Windows side dependency configuration and Maven dependency configuration. The last is the actual operation, including: obtaining the remote hadoop hdfs connection and a series of operations on it, including; Folder creation, file upload, file download, f ...

Introduction: This article is about the first process of big data offline data processing project: data collection Main contents: 1) Use flume to collect website log file data to access.log 2) Write shell script: split the collected log data file (otherwise the access.log file is too large) and rename it to access_ Mm / DD / yyyy.log. &nbsp ...

1. Introduction to sqoop (1) Introduction: Sqoop is a tool of Apache for "transferring data between hadoop and relational database server".   import data: import data from MySQL and Oracle to hadoop's hdfs, hive, HBASE and other data storage systems.      Export data: export data from hadoop file system to relation ...

^_^ 1. Configure local hadoop Hadoop 2.7.5 link: https://pan.baidu.com/s/12ef3m0CV21NhjxO7lBH0Eg Extraction code: hhhh Unzip the downloaded hadoop package to disk D for easy search Then right-click the computer and click Properties → click Advanced system settings on the right → click environment variables → select the Path b ...