Programmer Think

premise stay CDH It must be ensured before installation NODE1 The following programs in the master node have been started 1.Each machine :according to aliyun Time synchronization of services provided :ntpdate -u ntp6.aliyun.com Start service systemctl start ntpd systemctl restart ntpd Check to see if it starts ps -ef | grep n ...

When we use the Linux system, if there is a problem with the network or disk I/O, we will find that the process is stuck. Even using kill -9 can't kill the process. Many commonly used debugging tools, such as strace and pstack, also fail. What's the matter?At this point, we use ps to view the process list. We can see that the status of the stuc ...

1. Data analysis 1.1 problems This case requires statistical analysis exercises: Use the client to create the input directory on hdfsAnd upload * txt file to input directoryCall the cluster to analyze the uploaded files and count the words with the most occurrences 1.2 steps To implement this case, you need to follow the following st ...

1. Deploy Hadoop 1.1 problems This case requires the installation of stand-alone Hadoop: Hot word analysis:Minimum configuration: 2cpu, 2G memory, 10G hard diskVirtual machine IP: 192.168.1.50 Hadoop 1Installing and deploying hadoopData analysis to find the most frequently occurring words 1.2 steps To implement this case, you need to ...

catalogue 1, Entry class - SparkSubmit 2, SparkApplication startup - JavaMainApplication, YarnClusterApplication 3, SparkContext initialization 4, YarnClientSchedulerBackend and YarnClusterSchedulerBackend initialization 5, ApplicationMaster startup 6, Spark on Yan task submission process summary 1, Entry class - SparkSubmit When sub ...

After learning Hadoop for a semester, I finally chewed off this big bone, tears!!! This article is more like a summary of learning Hadoop 1, Preparatory work 1. hadoop compressed package There will be this official website. Download the compressed package and prepare it. I use version 2.7.1 2. jdk compressed package This is the Java r ...

BUG BUG is written in front: Kerberos 1.15 1-18. el7. x86_ Version 64 has a BUG, do not install this version!!!! If you have installed the version described above, don't be afraid. Here is a solution Upgrade kerberos 1. System environment 1. Operating system: CentOS Linux release 7.5 1804 (Core) 2. CDH: 5.16.2-1.cdh5.16.2.p0.8 3. Kerberos: 1 ...

catalogue 1, Introduction to azkaban 2, System architecture of azkaban 3, Installation mode of azkaban 3.1 Solo Server installation 3.1. 1 Introduction to solo server 3.1. 2 installation steps 3.2 installation method of multi exec server 3.2. 1 node layout 3.2. 2. Configure mysql 3.2. 3. Configure web server 3.2. 4. Configure exec se ...

Flume log collection tool is mainly used since it is a tool! Distributed acquisition processing and aggregation streaming framework A tool for collecting data by writing a collection scheme, that is, a configuration file. The configuration scheme is in the official document 1. Flume architecture Agent JVM process Source: receive data ...

Introduction: This is a learning note blog about the installation and deployment of flume. The main contents include: flume installation and deployment and two entry cases of flume. They are: the official case of monitoring port data and the file changes tracked by multiple files in the specified directory in real time. If there are mistakes, p ...