Discrete Cosine Transform [DCT]

Description: Binarizer is a Transformer.

Discrete cosine transform is a kind of transform related to Fourier transform. It is similar to discrete Fourier transform but only uses real numbers. The discrete cosine transform is equivalent to a discrete Fourier transform which is about twice its length. This discrete Fourier transform is carried out for a real even function (because the Fourier transform of a real even function is still a real even function).

Program example:

def getDataFrame(sparkSession: SparkSession = this.getSparkSession()): DataFrame = {
     sparkSession.createDataFrame(Seq(
          (0, Vectors.dense(0.0, 1.0, -2.0, 3.0)),
          (1, Vectors.dense(-1.0, 2.0, 4.0, -7.0)),
          (2, Vectors.dense(14.0, -2.0, -5.0, 1.0))
     ))
     .toDF("id", "features")
 }

def execute(dataFrame: DataFrame) = {
    //Feature name
    var feature = "words"
    var feature_new = "words_count_vectorizer"
    //Set up model
    val dct = new DCT()
    .setInputCol("features")         //Features to be transformed
    .setOutputCol("features_dtc")    //Transformed feature name
    .setInverse(false)               //true: execute anti DCT,false: execute forward DCT. Default: false
    //Model test
    var transform = dct.transform(dataFrame)
    //show
    transform.show(100, 100)
    
    dataFrame.show(false)
}

Data results:

+---+--------------------+----------------------------------------------------------------+
| id|            features|                                                    features_dtc|
+---+--------------------+----------------------------------------------------------------+
|  0|  [0.0,1.0,-2.0,3.0]|[1.0,-1.1480502970952693,2.0000000000000004,-2.7716385975338604]|
|  1| [-1.0,2.0,4.0,-7.0]|  [-1.0,3.378492794482933,-7.000000000000001,2.9301512653149677]|
|  2|[14.0,-2.0,-5.0,1.0]|   [4.0,9.304453421915744,11.000000000000002,1.5579302036357163]|
+---+--------------------+----------------------------------------------------------------+

Practical application example:

Discrete cosine transform, often used in signal processing and image processing, is used for lossy data compression of signals and images (including still images and moving images)