Programmer Think

Installation and deployment of Flink First, open the terminal and execute the command under / home/xxx / to obtain the installation package: wget https://labfile.oss.aliyuncs.com/courses/3423/flink-1.10.0-bin-scala_2.11.tar tar -xvf flink-1.10.0-bin-scala_2.11.tar Unzip it to this directory. If you have studied Spark, you should feel that t ...

Spark BigData Program: big data real-time stream processing log 1, Project content Write python scripts to continuously generate user behavior logs of learning websites.Start Flume to collect the generated logs.Start Kafka to receive the log received by Flume.Use Spark Streaming to consume Kafka's user logs.Spark Streaming cleans the data ...

The most detailed Hive article series in the whole network, it is strongly recommended to collect and pay attention! Later, the updated articles will list the catalogue of historical articles to help you review the key points of knowledge. catalogue Series of historical articles preface Hive database and table operations 1, Database op ...

Mapper class // // Source code recreated from a .class file by IntelliJ IDEA // (powered by Fernflower decompiler) // package org.apache.hadoop.mapreduce; import java.io.IOException; import org.apache.hadoop.classification.InterfaceAudience.Public; import org.apache.hadoop.classification.InterfaceStability.Stable; @Public @Stable public cla ...

brief introduction Using Pandas's pivot method, DF can be rotated and transformed. This paper will explain the secret of pivot in detail. Using Pivot pivot is used to reorganize the DF, and use the specified index, columns and values to reconstruct the existing DF. Take a Pivot example: Through pivot change, the new DF uses the value ...

11.1 introduction to experimental environment The cluster environment is running normallyHive and Impala services are installed in the clusterOperating system: redhat6 fiveCDH and CM versions are 5.11.1EC2 user with sudo permission is used for operation 11.2 UDF function development - using Intellij tools Use Intellij tool to develop Hive's ...

Introduction: use IK word splitter to segment words and generate word cloud.This paper mainly introduces how to make word frequency statistics through IK word segmentation. Use the word splitter to count the word frequency of the article. The main purpose is to realize the word cloud function as shown in the figure below, and you can find the k ...

When query time word segmentation and storage time word segmentation are not set in es Library 1, Index with default settings Picture: 1. When querying es using MatchQueryBuilder @Test public void searchTest() throws IOException{ // SearchResponse response = service.search("name", "shoes-0-num", 0, 30 ...

Introduction: use IK word splitter to segment words and generate word cloud. This paper mainly introduces how to make word frequency statistics through IK word segmentation. Use the word splitter to count the word frequency of the article. The main purpose is to realize the word cloud function as shown in the figure below, and you can find the ...

In the actual use process, we sometimes want to import and search RSS data. In many real micro services, a lot of data is provided in the form of RSS feed, such as our common comment websites. So is there any way to import these data into Elasticsearch and search? The answer is to use the information provided by Logstash RSS input plugin . In ...