Big data -- sorting out common interview questions in Scala

Posted by cedricm on Fri, 19 Nov 2021 22:13:09 +0100

catalogue

1. Advantages of Scala language

2. Closures in Scala

3. Coritization in Scala

4. The relationship and difference between Java and Scala

5. Pattern matching in Scala

6. Difference between case class and class

7. Talk about implicit operation in Scala

8. Partial function and partial application function in Scala

9. Tuples in Scala

10. The difference between trait and abstract class

11. Difference between arraybuffer and Array

1. Advantages of Scala language

  • Scala is a multi paradigm programming language, which is designed to integrate various features of object-oriented programming and functional programming
  • Scala runs on the Java virtual machine and is compatible with existing Java programs
  • Scala source code is compiled into Java bytecode, so it can run on the JVM and call existing Java class libraries
  • As the source code programming language of Spark, a popular open source big data memory computing engine, Spark has good performance advantages
  • Scala will become the mainstream language for big data processing in the future

2. Closures in Scala

         Definition: you can define a function, package, class, or even another function or method within any scope. In the function body, you can access any variable in the corresponding scope. (important) functions can be called when variables are no longer in scope.

object Test {
 
  /**
    * scala Closure in
    * Functions can also access variables when they are not in their valid scope
    *
    * @param args
    */
 
  def main(args: Array[String]): Unit = {
 
    def getHelloFunc(msg:String) = (name:String) => println(msg + "," + name)
 
    val sayHello = getHelloFunc("hello") // Pass in the value of msg, and sayHello can still access it later
    val sayHi = getHelloFunc("hi")
 
    // Use gethellofenc twice to pass in different msg and create different function returns
    // However, msg is only a local variable,
    // After gethellofenc is called, it still exists in the created function. When sayHello("yxj") is called, the msg with the value of hello is kept inside the function and can be used repeatedly
    // This variable is beyond its scope and can also be used, that is, closure
 
    // scala
 
    sayHello("yxj") // Print hello,yxj
    sayHi("yxj") // Print hi,yxj
 
  }
}

3. Coritization in Scala

         Coriolis function, also known as multi parameter list function, itself refers to the technology of changing the function that accepts multiple parameters into a function that accepts a single parameter (the first parameter of the initial function) and returns a new function that accepts the remaining parameters and returns the result

        A simple understanding is to change the form of the function without changing the function of the function. Usage: combined with implicit parameters, it greatly simplifies the code, but reduces the readability of the code

//This function takes two arguments
def mul(x:Int,y:Int) = x*y

//This function takes one parameter and generates another function that takes a single parameter
def mulOneAtTime(x:Int)(y:Int) = x*y

//It can also be written as
def mulOneAtTime(x:Int) = (y:Int) => x * y

4. The relationship and difference between Java and Scala

Contact:

  • Scala comes from Java, but it is higher than Java. Scala adds functional programming on the basis of Java, so that developers can develop programs through functional programming
  • Since Scala is finally compiled as a. class file and runs in the JVM, it is still Java in essence, so the API s of both sides can be interchanged in scala and Java

difference:

  • Declaration of variables:  
    • Variable var constant val, Scala supports automatic type inference
    • Scala uses constants rather than variables to solve problems. The advantage is that it can reduce the security problems of multithreading concurrency. It is especially suitable for multi concurrency distributed scenarios
  • Declaration of function
    • Keyword def, Scala function has no return value, Unit is used, which is equivalent to void in Java
    • Scala supports functional programming and can use high-order functions. Functions are first-class citizens in scala
  • Basic type
    •   Scala is a quintessential object-oriented programming language, so there is no real basic type. Its types are classes
  • static state
    • static in Java is contrary to the object-oriented programming idea and encapsulation characteristics of Java
    • Scala cancels the concept of static and implements it with a singleton object
  • Support for Strings
    • Scala supports the use of string difference to format strings, starting with $
    • In addition, it supports the use of three quotation marks to wrap the contents directly, which can include any character without escape
  • class
    • Fields in Scala are automatically provided with getter and setter methods. In addition, the @ BeanProperty annotation can be used to generate Get/Set methods in Java
    • Every class in Scala has a main constructor, which is "intertwined" with the class definition. The parameters of the class directly become the fields of the class. The main constructor executes all the statements in the class body
  • break is not supported in Scala
    • Use return instead
    • Guard with if and Boolean variables in the loop
    • Import packages that support break in Java
  • Access scope issues
    • In Java, you can't see the inside from the outside, but you can see the outside from the inside
    • In Scala, the outside cannot see the inside, and the inside cannot see the outside
  • wildcard
    • Java uses * for general configuration
    • Scala uses_ Carry out general distribution
  • Default imported package
    • java.lang package, Scala package and Scala.Predef class are imported by default in scala
    • Java imports the java.lang package by default
  • Trait trait can be compared with the interface in Java, but it is very different from the interface
    • In Java, classes implement interfaces, and in Scala they are called mixed attributes
    • Unlike interfaces in Java, features in Scala can include methods with method bodies

5. Pattern matching in Scala

        Scala's pattern matching includes a series of alternatives. Each alternative is in keyword case. Each alternative includes one pattern or multiple expressions. If it matches, it will be calculated. The arrow symbol = > separates the pattern from the expression

         Pattern matching in Scala is similar to the switch syntax in Java, but more powerful. In the pattern matching syntax, the match keyword is used to declare, and each branch is declared with the case keyword. When matching is needed, it will start from the first case branch. If the matching is successful, the corresponding logic code will be executed. If the matching is unsuccessful, continue to execute the next branch for judgment. If all cases do not match, the case is executed_ Branch, similar to the default statement in Java.

 obj match{

        case 1 => "one"

        case 2 => "two"

        case 3 => "three"

        case _ => default

      }

6. Difference between case class and class

case class is a sample class:

         The sample class is a syntax sugar of an immutable and decomposable class, that is, some syntax sugars will be automatically generated during construction. It has the following characteristics:

  • Automatically add the constructor consistent with the class name (that is, the half life object, which is implemented by the apply method), that is, the new keyword is not required when constructing the object
  • The parameter in the sample class defaults to the val keyword and cannot be modified
  • The toString, equals, hashcode and copy methods are implemented by default
  • The sample class can compare two objects by = =. If not in the construction method, the second attribute will not be used for comparison

Class is a class:

class needs to use the new keyword when constructing objects.

7. Talk about implicit operation in Scala

(1) Implicit variable

         Provide implicit parameters for functions (often used in conjunction with Coriolis)

def method(name:String)(prefix:String="DEFAULT") = print(prefix + name)

implicit prefix:String="DEFAULT"
def method(name:String)(implicit prefix:String) = print(prefix + name)

(2) Implicit function

        Provide default values for parameters of function types,

         Only one type can appear in the same scope

          Applying: implicit type conversions

def main(args: Array[String]): Unit = {
    //Implicit function, which converts string type to integer type by default
    //Only one of the same type can appear in the same scope, that is, the string type can no longer be implicitly converted to other types in the scope
    implicit def fsi(v:String):Option[Int] = {
        val regexInt = "\\d+".r
        if (regexInt.pattern.matcher(v).matches){
            Some(v.toInt)
        }else{
            Option.empty
        }
    }
    val a:Option[Int]="abc"
    println(a)
}

(3) Implicit class

         Implicitly extends the function of the type specified by the unique construction parameter

//Function: if the array type is Int, sum, otherwise output the string with a space as the spacer	  
implicit class ArrExt[T](arr:Array[T]){
	    def c()={
	      if (arr.isInstanceOf[Array[Int]] ){
	        arr.asInstanceOf[Array[Int]].sum
	      }else if(arr.isInstanceOf[Array[String]]){
	        arr.asInstanceOf[Array[String]].mkString(" ")
	      }else{
	        throw new RuntimeException("unsupported array type")
	      }
	    }
	  }
	  def main(args: Array[String]): Unit = {
	    println(Array(1, 2, 5).c())
	    println(Array("aa", "bb", "cc").c())
	  }

(4) Implicit object

         Extended function

//Function: automatically implement the function of the corresponding type according to the passed in parameter value type
 def main(args: Array[String]): Unit = {
	    trait A[T]{
	      def combine(arr:Array[T]):T
	      def empty:T
	    }
	    implicit object StrArrA extends A[String] {
	      override def combine(arr: Array[String]): String = arr.mkString(",")
	      override def empty: String = ""
	    }
	    implicit object IntArrA extends A[Int] {
	      override def combine(arr: Array[Int]): Int = arr.sum
	      override def empty: Int = 0
	    }

	    def combine[T](arr:Array[T])(implicit a:A[T]) = {
	      if (arr.isEmpty){
	        a.empty
	      }else{
	        a.combine(arr)
	      }
	    }
	    val arrStr: Array[String] = Array("aa", "bb", "cc")
	    val arrInt: Array[Int] = Array(1, 2, 3)
	    println(combine(arrStr))
	    println(combine(arrInt))
	  }

8. Partial function and partial application function in Scala

Partial function

         It is a mathematical concept. It is not a kind of "function". It is parallel to function. Partia Function in Scala is a trail of PartialFunction[A,B], which receives a parameter of type A and returns a result of type B.

        It is usually used in combination with model matching

scala> val pf:PartialFunction[Int,String] = {
     |   case 1=>"One"
     |   case 2=>"Two"
     |   case 3=>"Three"
     |   case _=>"Other"
     | }
pf: PartialFunction[Int,String] = <function1>
scala> pf(1)

res0: String = One


scala> pf(2)

res1: String = Two


scala> pf(3)

res2: String = Three


scala> newPF(4)

res3: String = Other

         There are some methods inside partial functions, such as isDefinedAt, OrElse, andThen, applyOrElse, and so on.

Partial applied function

         It is also called partial application function, which is only one word from the English name of partial function, but there is a big difference between them.

         Partial application function means that a function has n parameters, and we provide it with less than n parameters, then we get a partial application function, similar to coritization

scala> def add(x:Int,y:Int,z:Int) = x+y+z
add: (x: Int, y: Int, z: Int)Int
scala> def addX = add(1,:Int,:Int) // x known

addX: (Int, Int) => Int


scala> addX(2,3)

res1: Int = 6


scala> addX(3,4)

res2: Int = 8


scala> def addXAndY = add(10,100,_:Int) // x and y are known

addXAndY: Int => Int


scala> addXAndY(1)

res3: Int = 111


scala> def addZ = add(:Int,:Int,10) // z known

addZ: (Int, Int) => Int


scala> addZ(1,2)

res4: Int = 13

9. Tuples in Scala

  • Tuples in Scala are a fixed number of combinations, and ontologies can be passed as a parameter
  • Tuples can hold different types of data
  • Tuples are immutable
  • Tuples have an upper limit and can have up to 22 elements

10. The difference between trait and abstract class

  • A class can only inherit one abstract class, but it can inherit multiple attributes through the with keyword
  • Abstract classes have constructors with parameters, but the characteristics are not good (such as trait t(i: Int) {}, this declaration is wrong).
  • It is convenient for a class to extend multiple characteristics, but it can only extend one abstract class. So give priority to traits. If constructor parameters are required, use abstract classes  

11. Difference between arraybuffer and Array

  • Array can put numbers, strings, Boolean values, objects and arrays. ArrayBuffer can put binary data composed of 0 and 1
  • Array is stored in the heap, while ArrayBuffer stores the data in the stack (so the latter is faster when fetching data)
  • After the ArrayBuffer is initialized, the array size is fixed, and the array can be increased or decreased freely (in fact, a new array is generated)

Sorting of previous interview questions:

Big data -- sorting out common interview questions in Java 

Big data - sorting out common interview questions of Hadoop  

Big data - Hive common interview questions sorting  

Big data - collation of common interview questions in HBase  

Big data - sorting out Flink common interview questions

Topics: Scala Big Data