Flink tutorial (12) - Flink advanced API (Time and Watermaker)
01 introduction
In the previous blog, we have a certain understanding of the use of Flink batch streaming API. Interested students can refer to the following:
Flink tutorial (01) - Flink knowledge mapFlink tutorial (02) - getting started with FlinkFlink tutorial (03) - Flink environment constructionFlink tutorial (04) - getting started wi ...
Posted by cwls1184 on Mon, 07 Mar 2022 16:36:49 +0100
Resource scheduling in Yarn
Three Scheduling Strategies
FIFO Scheduler, Capacity Scheduler and Fair Scheduler policies are listed from left to right. These three policies are introduced below
FIFO Scheduler: first in, first out scheduling strategy Tasks are carried out in turn. Resources can only be released after the execution of previous tasks. This is unreasona ...
Posted by ConnorSBB on Sat, 08 Jan 2022 04:07:34 +0100
spring boot integrates spark and runs and submits spark task spark on yarn based on yarn
preface
The previous project was based on springboot and integrated spark, running on standalone. I once wrote a blog, link:
https://blog.csdn.net/qq_41587243/article/details/112918052?spm=1001.2014.3001.5501
The same scheme is used now, but spark is submitted on the production environment yarn cluster, and kerbores verification is required, ...
Posted by not_john on Mon, 03 Jan 2022 23:15:07 +0100
Yan web page port 8088 is not accessible in Windows browser
Problem description
After configuring the Hadoop Namenode, secondarynode, and datanode nodes, configure mapred site in the hadoop /etc directory xml,yarn-site.xml file. After configuration, start resource manager and nodemanager. Among them, yen site The XML configuration is as follows:
<property>
<name>yarn.node ...
Posted by matbennett on Sun, 19 Dec 2021 13:56:35 +0100