Programmer Think

Spark Day11: Spark Streaming 01 - [understand] - yesterday's course content review Main explanation: Spark Streaming module quick start 1,Streaming Overview of flow computing - Streaming Application scenario Real time report RealTime Report Real time increment ETL Real time early warning and monitoring Real time search recomm ...

1. Introduction to sqoop (1) Introduction: Sqoop is a tool of Apache for "transferring data between hadoop and relational database server".   import data: import data from MySQL and Oracle to hadoop's hdfs, hive, HBASE and other data storage systems.      Export data: export data from hadoop file system to relation ...

Overall framework Spark storage adopts master-slave mode, i.e. Master / Slave mode. The whole enclosure uses RPC message communication mode. Of which: The Master is responsible for the management and maintenance of data block metadata during the operation of the whole applicationSlave is responsible for reporting the status information of ...

We may often use Hibernate, Mybatis, jpa and other frameworks in our study and work. These frameworks have a good encapsulation of the database connection pool and may ignore the underlying implementation of the database. Today, let's take a look at how to write a simple database connection pool. Before that, let's recall the steps of ja ...

        Redis supports a variety of data structures, such as string, list, set, ordered set and hash. This time I sorted out about   Hash   Related commands, that is, about   Hashes   The related commands are as follows.         The part in the red circle in the figure above is ...

Cdh6.3.2 impala 3.2 to impala 3.4 compilation process Local environment hardware requirements The CPU must support at least SSSE3 Minimum memory: 16GB (64G recommended by the community) Hard disk space: 120GB (for test data) Linux only operating systems Ubuntu 14.04,16.04,18.04 CentOS 7 Compiling environment A set of CDH6.3.2 clus ...

1, How many constraints do you master in MYSQL? In MYQSL, there are some constraints. For example, it cannot be null or must be unique. Like ID cards, everyone's ID number is unique. Gender can be male or female, but not null. The following describes six constraints in MYSQL 1. NOT NULL: a constraint column cannot be empty. -- Cre ...

Chapter 3 DStream creation 3.1 RDD queue 3.1.1 usage and description During the test, you can create a DStream by using ssc.queueStream(queueOfRDDs). Each RDD pushed to the queue will be processed as a DStream. 3.1.2 case practice Requirement: create several RDD S in a loop and put them into the queue. Create Dstream through SparkSt ...

catalogue 1. Advantages of Scala language 2. Closures in Scala 3. Coritization in Scala 4. The relationship and difference between Java and Scala 5. Pattern matching in Scala 6. Difference between case class and class 7. Talk about implicit operation in Scala 8. Partial function and partial application function in Scala 9. Tuples in Sc ...

JAVA----API 1, API 1. Basic use of scanner API Explanation: The full name is application programming interface(Application Programming Interface), Originally meant to mean JDK Various classes and interfaces provided For example: Scanner Get string public String nextLine(); Get the string entered by the user and get the whole line of data. ...