Programmer Think

3, RDD action operator The so-called action operator is actually a method that can trigger Job execution 1,reduce  function signature def reduce(f: (T, T) => T): T  function description Aggregate all elements in RDD, first aggregate data in partitions, and then aggregate data between partitions val rdd: RDD[Int] = sc.m ...

A FEW MISCELLANEOUS ITEMS https://docs.scala-lang.org/overviews/scala-book/misc.html In this section we'll cover a few miscellaneous items about Scala: Tuples tuplesScala OOP example of pizza restaurant order entry system TUPLES https://docs.scala-lang.org/overviews/scala-book/tuples.html Tuple is a concise class that provid ...

Basic grammar The pattern matching in scala is similar to the switch syntax in Java, but Scala adds more functions from the syntax, so it is more powerful. In the pattern matching syntax, the match keyword is used to declare, and each branch is declared with the case keyword. When matching is needed, it will start from the first case branc ...

Factory mode First, let's look at the traditional design pattern Now there is a demand: the production of Xiaomi mobile phones Mobile interface //Mobile interface public interface Phone { void make(); } Xiaomi mobile phone //Xiaomi mobile phone public class MiPhone implements Phone { public MiPhone() { this.make(); } ...

Preface I have read several foreign blogs about Java map in the last few days and have written very well, so I have sorted out eight issues that Java map should master, which are common problems in daily development. I hope they will help you. If there are any incorrect points, please mention them. Thank you very much. 1. How to convert a Map ...

1. Origin of functional interface We know that the premise of using Lambda expression is to have functional interface, and Lambda expression does not care about interface name and abstract method name. Only the parameter list and return value type of the abstract method are concerned. Therefore, in order to make it easier for us to use Lam ...

Error log: 1. Error log: ​ WARN TaskSetManager: Lost task 29.0 in stage 22.0 (TID 1851, wn108-cdlcns.bjduloineequ3adfbkrpgi4p2c.shax.internal.chinacloudapp.cn, executor 25): FetchFailed(BlockManagerId(2, wn107-cdlcns.bjduloineequ3adfbkrpgi4p2c.shax.internal.chinacloudapp.cn, 7447, None), shuffleId=12, mapId=4, reduceId=29, message= org.apach ...

1. Array 1.1 general Array is a container used to store multiple elements of the same type. Each element has a number (also known as subscript, subscript and index), and the number starts from 0. In Scala, there are two kinds of arrays, one is a fixed length array and the other is a variable length array 1.2 fixed length array 1.2.1 feature ...

Overall framework Spark storage adopts master-slave mode, i.e. Master / Slave mode. The whole enclosure uses RPC message communication mode. Of which: The Master is responsible for the management and maintenance of data block metadata during the operation of the whole applicationSlave is responsible for reporting the status information of ...

catalogue 1. Advantages of Scala language 2. Closures in Scala 3. Coritization in Scala 4. The relationship and difference between Java and Scala 5. Pattern matching in Scala 6. Difference between case class and class 7. Talk about implicit operation in Scala 8. Partial function and partial application function in Scala 9. Tuples in Sc ...