Higher order function

• The so-called high-order function is actually to use the function as an object;
• Functions also have types, that is, function types

1 function is assigned to the variable as a value

1.1 assigning a parameterless function object to a variable

1. Phenomenon introduction

object Scala05_Function_Hell {
  def main(args: Array[String]): Unit = {
    //todo 1. Assign a function as a value to a variable
    def test() : Unit = {
      println("f1....")
    }

    // Question 1: try to assign the function object test to the variable f, and then execute the function through f
    val f = test
    f()
    // Execution error
  }
}

2. Reasons

• If the parameter list declared by the function has no parameters, the parentheses can be omitted when calling
  Therefore, the following two expressions are equivalent

    val f = test
    val f = test()

Since f is the execution result of the method, f is not a function and cannot f()
3. Correct writing

object Scala05_Function_Hell {
  def main(args: Array[String]): Unit = {
    //todo 1. Assign a function as a value to a variable
    def test() : Unit = {
      println("f1....")
    } 
  
    // Use special symbols to declare, assign a function to a variable as an object,
    // Instead of having this function execute
    val f = test _
    f()
  }
}

For nonparametric functions, if you assign them to variables, you need to use_ Assign a function object to a variable;
Otherwise, there will be ambiguity, because the call of parameterless function can not add ()
4. Verification

object ScalaFunction {
    def main(args: Array[String]): Unit = {
        def fun1(): String = {
            "zhangsan"
        }
        val a = fun1
        val b = fun1 _
        println(a)
        println(b)
    }
}

a is the result returned by the function
b is the function object

1.2 type of function object

Type of variable f: () = > unit
Definition method of function type: in = > out

• When we assign a function object to a variable, if we don't want ambiguity caused by type inference, we can directly specify the function type when declaring the variable; As follows:

	val f = test _
    f()

    val f1 : ()=> Unit = test
    f1()

Both ways are OK!

The second way to write a function object type

	val a : () => String = test _
	val a : Function0[String] = test _

There are only 22 types of function objects in Scala, from Function0 to Function22; Therefore, when a function is passed to a variable as an object, the parameters of the function can only be 22; If a function is used as a normal function, there are no parameter restrictions

1.3 assign a function object of a parameter to a variable

object Scala05_Function_Hell1 {
  def main(args: Array[String]): Unit = {

      def test(i:Int) : Int = {
        i*2
      }

    //todo passes the value of a function object to a variable to solve two problems:
    // 1. How to pass function objects to variables
    // 2. How is the type of function object defined
     val f : (Int)=>Int = test
     println(f(1))

    // In function types, parentheses can be omitted when there is only one function parameter
    // However, when a function has no parameters, parentheses should not be omitted
    val f1 : Int =>Int = test
  }
}

• There are two problems to be solved when passing the value of a function object to a variable:
  1. How to pass function objects to variables
  2. How is the type of function object defined

• Definition of function object type:
  1. Parentheses can be omitted when there is only one function parameter
  2. When the function has no parameters or has more than one parameter, do not omit the parentheses

1.4 assign function objects of multiple parameters to variables

    def test1(name :String,password:String) : String = {
      name + "," + password
    }

    val f2 : (String,String)=>String = test1

2 function as parameter

object Scala05_Function_Hell2 {
  def main(args: Array[String]): Unit = {

    //todo passes parameters as parameters
    def test(f: (String) => String ) : String = {
         f("Zhang San")
    }

    //Define function
    def f(s : String) : String = {
      s * 2
    }

    //Function object
    val v = f _
    // Function object passed in
    println(test(v))
    println(test(f _))
    println(test(f))
  }
}

There are two ways:

1. Function object_ Make parameters
2. Function object as parameter
   Since the parameter types in the main function have been defined, you can directly pass the function object as the parameter

2.1 simplification principle of anonymous function parameters

The function name is irrelevant, and anonymous functions can be passed directly

object Scala05_Function_Hell2 {
  def main(args: Array[String]): Unit = {

    //todo passes parameters as parameters
    def test(f: (String) => String ) : String = {
         f("Zhang San")
    }

    // todo 2. Anonymous function as parameter
    val a = (s: String) => {s * 2}
    println(test(a))

    // todo 2.1 anonymous function takes parameters directly
    test( (s: String) => {s * 2} )

    // todo 2.2 can infer the type of parameter (anonymous function object) according to the parameters of the main function, so the parameter type declaration of anonymous function can be omitted
    test( (s)=>{s * 2} )
    // todo 2.3 if the anonymous function parameter list has only one, omit the parentheses
    test( s => {s * 2})
    // todo 2.4 if the anonymous function method body has only one line of logic, braces can be omitted
    test( s => s * 2 )
    // todo 2.5 if the parameter of an anonymous function is used only once, you can omit the parameter, write the method body directly, and replace the parameter with an underscore
    test( _ * 2)
  }
  }

• Simplification principle of anonymous function parameters
  1. The type of the parameter (anonymous function object) can be inferred from the parameters of the main function, so the parameter type declaration of the anonymous function can be omitted
  2. If there is only one anonymous function parameter list, omit the parentheses
  3. If the anonymous function method body has only one line of logic, braces can be omitted
  4. If the parameter of an anonymous function is used only once, you can omit the parameter, write the method body directly, and replace the parameter with an underscore
  (tips: the parameter is used only once, which means that it is used only once in the method body; If the parameter is used more than once, or the order in which the parameter is used does not correspond to the order in which it is declared, it cannot be replaced by an underscore)

2.2 application of function as parameter

• The main function provides parameters, and logic
• The logic is provided by additional functions. This function is passed into the main function as a parameter to inject logic into the main function
• The parameters and logic of the main function are available, so it can be executed

object Scala05_Function_Hell3 {
  def main(args: Array[String]): Unit = {
    def test(x:Int,y:Int,f: (Int,Int) => Int) = {
        f(x,y)
    }

    test( 1, 5, (x:Int,y:Int)=>{x+y} )
    test( 1,5,  (x,y)=>{x+y} )
    test( 1,5,  (x,y)=>x+y )
    test( 1, 5, _ + _)
  }
}

3 function as return value

object Scala05_Function_Hell4 {
  def main(args: Array[String]): Unit = {
    //todo 1. The first method: do not actively declare the return function type, but the returned function object needs to be added_
    // This method is recommended
    def test() = {
      def inner() : Unit = {
        println("xxxxxxx")
      }
      inner _
    }
    //todo 2. The second method: actively declare the return function type, and the returned function object does not need to be added_
    def test1() : ()=>Unit = {
      def inner() : Unit = {
        println("xxxxxxx")
      }
      inner
    }

    //todo 3. Use
    val f = test()
    f()

    //todo 3.2 use
    test()()

  }

}

object Scala05_Function_Hell5 {
  def main(args: Array[String]) :Unit = {
      def outer(x:Int)  = {
          def middle(y:Int) ={
            def inner(f:(Int,Int)=>Int) ={
                f(x,y)
            }
            inner _
          }
        middle _
      }

    //use
    println(outer(10)(10)(_ + _))
  }
}

4. Closure

4.1 demonstration of stack relationship corresponding to function relationship

(1) The two methods are executed sequentially:

In this way, there is only one stack frame in the virtual machine stack at the same time; After test1() is executed, test2() is pushed in
(2) Method 1 contains method 2

In this case, there will be two stack frames in the virtual machine stack at the same time;

4.2 code demonstration closure

object Scala05_Function_Hell6 {
  def main(args: Array[String]): Unit = {
  
    def outer(x:Int) = {
      def inner(y:Int) = {
        x+y
      }
      inner _
    }

    println(outer(20)(20))
  }
}

In the code, this is the first case

• todo 1.Scala is based on Java, so functions must be methods in bytecode files
• todo 2. Parameter x should be a local variable of the method after compilation
  The scope of the local variable is internal to the current method
• todo 3.inner function is also a method in bytecode file
  The inner method is executed after the outer method is executed = > because the inner method is the execution result of the outer method
• todo 4. Questions
  After the outer method is executed, the stack frame pop-up local variable x will also be recycled
  At this point, the inner is stacked, so why can the inner use x?

Special operations are carried out at the bottom layer:

• If a function uses an external variable, the variable needs to change the life cycle
• At this time, the function includes this function into the function to form a closure environment

Decompile view:

The parameter x is passed to inner as a parameter, although inner has only one parameter when it is declared

4.3 implementation of closures

After Scala 2.12, closures are implemented by changing the parameter list of internal functions

Before Scala 2.12, closures were implemented by anonymous inner classes
Compile with scala2.11:

4.4 how to judge closure

How to determine whether there are closures?
Determine whether there are operations that change the function life cycle

Consider a question: can it be called closure without using external variables?

• All functions assigned to objects have closures
• All anonymous functions have closures

object Scala05_Function_Hell7 {
  def main(args: Array[String]): Unit = {
    def test() : Unit = {
      print("xxx")
    }

    // todo passes function objects to variables and also has closures
    //  A can be used as a return value to pass the function object to a place outside the scope  
    //  Local functions are put into external use,
    //  This changes the declaration period
    val a = test _
  }
}

In the code, the test() function itself is inside the method, and the scope is inside the method, but it is passed to variables. Variables can be passed outside the function in the form of return values. Therefore, the scope of the function object changes and the life cycle changes, so there are closures

5. Memory description

• Stack exception
  Stack overflow: too many stack frames
  Stack memory overflow: there are too many threads. One thread has one stack. There are many threads and many stacks, resulting in tight jvm memory

6. Control abstraction

We already know that you can pass a function as an object to another function; However, in some cases, you don't want to transfer functions, you just want to transfer logic; That is, instead of encapsulating logic into a function as a function object, write the method body directly; This means using part of the function instead of the whole function;

Only a part is abstract, and abstraction is incomplete. A method with only a declaration but no implementation is incomplete, which is an abstract method; An incomplete class is an abstract class; When there are abstract methods in a class, the class is also incomplete, so the class must be an abstract class; However, abstract classes do not necessarily have abstract methods

To control abstraction is to use incompleteness to control the content of abstraction

object Scala06_Function {
  def main(args: Array[String]): Unit = {
    //todo control Abstract Syntax:
    // Original writing: fun7 (F: () = > unit) = {...}
    // Control Abstract Writing: function object type in parameter: do not write parameter part
    //             Function object parameters in parameters: not written, replaced by op
    def fun7(op: => Unit) = {
      op  // op is equivalent to calling this function to execute the logic in the parameters
    }

    // todo call
    // Logic is passed in as a parameter
    // Also, since the logic is multiline, curly braces can be used instead of curly braces

    fun7{
      val i = 10
      val j = 20
      print(i+j)
    }

    // Control abstraction is: no function name, no function declaration, just logic!

  }
}

• Using control abstraction, a piece of code logic can be passed to other operations as parameters, so that the logic is not fixed and can change dynamically; Generally used to implement custom syntax

The break syntax in Scala uses control abstraction


breakable is a function;
The logic is passed in and executed in the try statement block. Once the break is executed, an exception is thrown, and the subsequent logic outside the for loop cannot be executed. Therefore, the exception is handled here, so the subsequent logic can be handled

7. Function coritization

object Scala07_Function_Curry {
  def main(args: Array[String]): Unit = {
    //Curry is a scientist

    val i = 100
    val j = 200
    def test(i:Int,j:Int):Unit={
      for(a <- 1 to i){
        println(a)
      }

      for(b <- 1 to j){
        println(b)
      }
    }

    // todo problem: the test method must have i and j parameters to execute
    //    However, in the test method, the code logic using i and the code logic using j are completely independent and irrelevant
    //    In other words, with i, you can execute the above for loop without waiting for j to pass in; Similarly j
    // Scientist Curry found that all method parameters must be prepared before calling the function
    // However, if there is no relationship between parameters, it will lead to performance degradation (waiting for both parameters to be ready) and increase business complexity;
    // Therefore, the parameters can be separated through the parameter list

    // todo function supports multiple parameter lists. Each parameter list represents a meaning, simplifying complex logic
    def test1(i:Int)(j:Int):Unit={
      i + j
    }
    // todo core points: in the function, the parameters are independent of each other, and the logic involved in the two does not interfere with each other

    // todo application
    // Separating value transfer from logic
    def operator(x:Int,y:Int)(op:(Int,Int)=>Int) : Int = {
      op(x,y)
    }

    operator(10,20)(_+_)

  }

}

8. Recursion

• The function calls itself in its logical processing.
• Recursive functions must have jump logic
• The parameters passed by recursion when calling should be regular

1. Problems with recursive functions

When the parameter is large, it will cause stack overflow

2. Tail recursion

Recursion with this circle identification tail;
StackOverflow will not appear in tail recursion

demonstration:


For the second figure, when test1() is executed, the child test1() will be pushed onto the stack first. After the child test1() is executed, because there is still code behind test1(), test1() cannot be pushed out of the stack;

For the first figure, test1() executes print first. When calling the sub test1(), it will be found that the elastic stack of test1() and the stack pressing (execution) of test1() are not affected; Therefore, you can play first and then press in, so there is always only one stack frame in the stack, so there will never be stack overflow

Tail recursion principle

Tail recursion in Scala language will be optimized as a while loop at compile time, so there will be no problem

Java also has tail recursion, but it will not be optimized when compiling Java. Scala will be compiled into Java when compiling

Although Java tail recursion will not be optimized and Stackoverflow will occur, the time of exception is later than that of ordinary recursion

The core of tail recursion: there is no relationship between stack out and stack pressing, which does not affect

8. Inertia function

When the return value of a function is given a variable declared lazy, the execution of the function will be delayed until we take this value for the first time. This function is called inert function.

object Scala09_Function_lazy {
  def main(args: Array[String]): Unit = {
    def fun9(): String = {
      println("function...")
      "zhangsan"
    }

//    val a = fun9()
//    println("----------")
//    println(a)
//    //Print results: sequential execution
//    //          function...
//    //          --------
//    //          zhangsan
      //Let's assume that the return value of fun9() is an object that takes up a lot of memory
      //So when a always references this object
      //Suppose the sentence println("----------------") is executed for 10min
      //This means that a large object has been occupying memory and has not been used
      //So it's been crowding out memory for a while

      //Can you wait until you use this object to get this object

      lazy val a = fun9()
      println("----------")
      println(a)
    //Print results:
    //          --------
    //          function...
    //          zhangsan

    //todo means that the function will be executed only when the function object is used, which is an inert function
  }
}

9. What exactly is a function

Why can't functions be used elsewhere, but methods can?
Method is called through an object
For example, the following function fun() cannot be used when calling test()

To view the code compiled into java:

The function is also a class method after compilation, but it is decorated with private static final;

• private class cannot be used outside
• final cannot be overridden. Function cannot be overridden in scope
• The function name $1 has changed; It is used to distinguish between methods and functions. If there is an external method called fun(), the name of the method will be duplicated and the method will not be renamed after compilation
  

What is a function?
Method of private static final in class

10. Basic principles of distributed computing

What if one machine wants another machine to perform a function?
It means distributing logic to other machines, not to objects

In the field of big data, the system architecture is master-slave architecture

Distributed computing has two cores:
1. Split the data to be calculated and calculate separately
2. The calculation logic is passed from the master to the worker

Principle of Distributed Computing:

• The data should be segmented according to the calculation node
• The calculation logic of the calculation node is dynamic and not fixed, so it is generally passed from the master to the worker
• After all calculation nodes have completed calculation, the calculation results can be returned to the master for consolidation to obtain the final results

This is also the core principle of Spark

Code implementation of distributed computing

class Task extends Serializable {
  // data
  val data = 1 to 5
  // Computational logic
  val logic = (x:Int) => x * 2
}

object Scala11_Function_Worker {
  def main(args: Array[String]): Unit = {
    //todo 1. Because the master sends data and calculation logic to the worker, the worker is the server side
    val server = new ServerSocket(9999)
    val master: Socket = server.accept()

    val objIn = new ObjectInputStream(master.getInputStream)
    val task : Task = objIn.readObject().asInstanceOf[Task]
    objIn.close()
    master.close()

    for(i <- task.data){
      println(task.logic(i))
    }
    println("The calculation is completed")
    server.close()
  }
}

···scala
object Scala11_Function_Master {
def main(args: Array[String]): Unit = {
val worker = new Socket("localhost",9999)
val objOut = new ObjectOutputStream(worker.getOutputStream)

val task = new Task()
objOut.writeObject(task)

objOut.flush()
objOut.close()
worker.close()

}
}
···