Programmer Think

The 20th day of learning big data - Collection and List (because you need to accompany your family for the new year, there is no time for learning to continue recording, and it will continue to be updated every day from today) Learning step by step according to the appeal map can get twice the result with half the effort aggregate The differ ...

Basic operation of SparkStreaming in PySpark preface Stream data has the following characteristics: • data arrives quickly and continuously, and the potential size may be endless • numerous data sources and complex formats • large amount of data, but do not pay much attention to storage. Once processed, it will either be discar ...

brief introduction The scenario introduces how to visually configure the data of student achievement through AnalyticDB, generate the large screen and dashboard of student achievement distribution with one click, and produce the achievement report by cycle through task arrangement. relevant Experience Lab Developers master what and how of cl ...

XML (eXtensible Markup Language) refers to eXtensible Markup Language, which is designed to transmit and store data. It has increasingly become the core of many new technologies and has different applications in different fields. It is the inevitable product of the development of web to a certain stage. It not only has the core characteristics ...

Introduction to azkaban and azkaban deployment, principles and usage Introduction to azkaban Azkaban is a simple task scheduling service that consists of three parts: web server, dbserver, executor server. Azkaban is a Java project from Linkedin Open Source, a batch workflow task scheduler. Used to run a set of work and processes in a specifi ...

kafka installation record: Official website: http://kafka.apache.org/downloads.html 1. Download yum install -y wget wget https://mirrors.bfsu.edu.cn/apache/kafka/2.6.0/kafka_2.12-2.6.0.tgz 2. Decompression: tar -zxvf kafka_2.12-2.6.0.tgz -C /opt/ 3. Change of name mv kafka_2.12-2.6.0 kafka 4. Create files cd kafka Create un ...

  Have you ever had time to fish In the Internet circle, it is often said that 996 work system, but there are also 965, especially 007, and 007 feels like an ICU. Therefore, everyone will sneak around and occasionally touch fish. There are many ways to fish. Have you ever fished at work? What did you do during your fishing time? If you fini ...

1. Introduction to data: Tianchi Taobao User Data 2. Data Import and Cleaning SQL interface tool used this time - SQL workbench 2.1 Importing data create table userbehavior( userID int, itemID int, categoryID int, bahaviortype text, timestamp int ); load data infile "C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/UserBehavior.csv" into ta ...

Mslab (memory local allocation buffer) of HBase preface This paper briefly introduces the write cache MemStore and data structure of HBase, as well as the function and source code analysis of MSLAB, which is the main component of write cache. MSLAB is the abbreviation of MemStore local allocation buffer. It carries out reasonable planning ...

1. Installation and deployment 1.1. Download source code https://github.com/azkaban/azkaban/tags 1.2. Decompression source code: tar -zxf azkaban-3.85.1.tar.gz -C /opt/soft 1.3. Compile source code You need to install GIT and Java environment # Build Azkaban ./gradlew build # Clean the build ./gradlew clean # Build and install dis ...