Programmer Think

catalogue 0. Links to related articles 1. Introduction to flinksql integration Hive 2. Basic ways of integrating Hive 2.1. Persistent metadata 2.2. Use Flink to read and write Hive's table 3. Preparation 4. SQL CLI 5. Code demonstration 0. Links to related articles Flink article summary 1. Introduction to flinksql integration Hive ...

3.2 RDD programming In Spark, RDD is represented as an object, and RDD is converted through method calls on the object. After defining RDD through a series of transformations, you can call actions to trigger the calculation of RDD. Actions can be to return results to the application (count, collect, etc.) or to save data to the storage system ...

Installation and deployment of Flink First, open the terminal and execute the command under / home/xxx / to obtain the installation package: wget https://labfile.oss.aliyuncs.com/courses/3423/flink-1.10.0-bin-scala_2.11.tar tar -xvf flink-1.10.0-bin-scala_2.11.tar Unzip it to this directory. If you have studied Spark, you should feel that t ...

catalogue 0. Links to related articles 1. Introduction to broadcaststate 2. Requirements - realize dynamic update of configuration 3. Coding steps 4. Code implementation 0. Links to related articles Flink article summary 1. Introduction to broadcaststate During the development process, if low throughput events such as distribution / b ...

Flink series Lesson 01: Flink's application scenario and architecture model Lesson 02: introduction to Flink WordCount and SQL implementation Lesson 03: Flink's programming model compared with other frameworks Lesson 04: Flink's commonly used DataSet and datastream APIs Lesson 05: Flink SQL & Table programming and cases Lesson 06: Flink cl ...

1. What is flick flink is a flow processing framework. Usually, the usage scenario is to consume kafka data and send it to other systems after grouping and aggregation. Grouping and aggregation are the core of flink. This paper only describes a single usage scenario. Stream data is equivalent to continuous data. The log data in kafka in produc ...

Flink1.12 - memory management 1. Foreword In order to enable users to better adjust memory allocation and achieve reasonable resource allocation, flynk1 10 introduces the memory management of TaskManager, which is later described in flink1 Version 11 introduces the memory management of JobManager. Users can allocate resources reasonably th ...

Catalog 0. Links to related articles 1. select operator 2. where operator 3. distinct operator 4. group by operator 5. union and union all operators 6. join operator 7. group window operator 7.1. tumble window operator 7.2. hop window operator 7.3. session window operator 0. Links to related articles Flink Article Summary 1. sele ...

DataStream common operators In the Flink application, whether your application is a batch program or a stream program, it is the model shown in the figure above. There are data sources and data sink s. The applications we write mostly do a series of operations on the data from the data source, which are summarized as follows. Source: da ...

1. Introduction to processfunction 1.1 description on API A function that processes elements of a stream. For every element in the input stream processElement(Object, ProcessFunction.Context, Collector) is invoked. This can produce zero or more elements as output. Implementations can also query the time and set timers through the p ...