Programmer Think

Introduction and description Four traditional mainstream databases:     Oracle MySQL SqlServer DB2 Non relational database: Redis MongoDB The mainstream database is relational database: there is an association relationship between tables When we say install database, we mean install database service When creating a database, it refers t ...

5.4 integrity constraint naming clause Integrity constraint naming clause Constraint < integrity constraint name > < integrity constraint >      ◾ < integrity constraints > include not null, unique, primary key phrase, foreign key phrase, check phrase, etc [example 5.1] establish a Student registration fo ...

Azkaban is a batch workflow task scheduler launched by Linkedin company. It is mainly used to run a group of work and processes in a specific order in a workflow. Its configuration is to set dependencies through simple < key, value > pairs and dependencies in the configuration. Azkaban uses job profiles to establish dependencies betwe ...

1 data warehouse concept Data Warehouse can be abbreviated as DW or DWH. Data Warehouse is a strategic set that provides all system data support for all decision-making processes of enterprises. The analysis of data in data warehouse can help enterprises improve business processes, control costs and improve product quality. Data warehouse is n ...

Hive supports the following formats for storing data: TEXTFILE (row storage), sequencefile (row storage), ORC (column storage) and PARQUET (column storage) 1: Column storage and row storage   The left side of the figure above is a logical table, the first one on the right is row storage, and the second one is column storage. The storage ...

1. Structure creation create Structure type structure name structure description; 2. Display structure show Structure type (plural) Display structure creation details: show create Structure type and structure name; 3. Data operation (data sheet) Add data: insert into Table name values View data: select from Table name Update data: updat ...

The DBLINK built this time is to access the SQL SERVER database on the Win side of the Damon database on the Linux side. By configuring ODBC to connect to the SQL SERVER database, DM creates an ODBC DBLINK connection to realize the DBLINK between DM and SQL SERVER. The following are the specific operation steps. DBLINK overview: DBLINK (Datab ...

Hive improved chapter Use of Hive Hive's bucket table 1. Principle of drum dividing table Bucket splitting is a more fine-grained partition relative to partition. Hive table or partition table can further divide bucketsDivide the bucket, take the hash value of the whole data content according to a column, and determine which bucket th ...

Introduction: Functions: clean the collected log data, filter invalid data and static resources Method: write MapReduce for processing Classes involved: 1) Entity class Bean Describe various fields of log data, such as client ip, request url, request status, etc 2) Tool class Used to process beans: set the validity or invalidity of log ...

catalogue The new architecture is integrated with the lake warehouse 1, Version Description 2, Compile and package Hudi version 0.10.0 1. Use git to clone the latest master on github 2. Compilation and packaging 3, Create a flick project 1. Main contents of POM document 2.checkpoint 3.flinkcdc code 4.hudi code (refer to the official ...