TensorFlow deep learning: 4.2. Mathematical operation of tensor

Posted by asaschool on Mon, 04 Oct 2021 05:04:55 +0200

Tensor mathematical operations mainly include scalar operations, vector operations and matrix operations. In addition, we will introduce the broadcast mechanism of tensor operation.

1. Scalar operation

The mathematical operators of tensors can be divided into scalar operators, vector operators and matrix operators.
Addition, subtraction, multiplication, division, power, trigonometric functions, exponents, logarithms and other common functions, logical comparison operators are scalar operators.
Scalar operators are characterized by element by element operations on tensors.
Some scalar operators overload common mathematical operators. It also supports numpy like broadcast features.
Many scalar operators are under the tf.math module.

a = tf.constant([[1.0,2],[-3,4.0]])
b = tf.constant([[5.0,6],[7.0,8.0]])
#The operator overload of mod is equivalent to m = tf.math.mod(a,3)
a%3 
>><tf.Tensor: shape=(3,), dtype=int32, numpy=array([1, 2, 0], dtype=int32)>
(a>=2)&(a<=3)
>><tf.Tensor: shape=(2, 2), dtype=bool, numpy=
>>array([[False,  True],
>>       [False, False]])>
x = tf.constant([2.6,-2.7])

tf.print(tf.math.round(x)) #Keep integer part, round = = [3 - 3]
tf.print(tf.math.floor(x)) #Keep integer part, round down = = [2 - 3]
tf.print(tf.math.ceil(x))  #Keep integer part, round up = = [3 - 2]
# Amplitude clipping
x = tf.constant([0.9,-0.8,100.0,-20.0,0.7])
y = tf.clip_by_value(x,clip_value_min=-1,clip_value_max=1)
#tf.clip_by_norm(t, clip_norm, axes=None, name=None)
#Similar to a normalization, the L2 norm along a dimension is less than clip_norm value
# * clip_norm / l2norm(t)
z = tf.clip_by_norm(x,clip_norm = 3)
tf.print(y)       ==>[0.9 -0.8 1 -1 0.7]
tf.print(z)       ==>[0.0264732055 -0.0235317405 2.94146752 -0.588293493 0.0205902718]

2. Vector operation

2.1.reduce

Vector operators operate only on a specific axis, mapping a vector to a scalar or another vector. Many vector operators begin with reduce.

#Vector reduce
a = tf.range(1,10)
tf.print(tf.reduce_sum(a))
tf.print(tf.reduce_mean(a))
tf.print(tf.reduce_max(a))
tf.print(tf.reduce_min(a))
tf.print(tf.reduce_prod(a))        ===>362880

Tensor specifies the dimension to reduce

b = tf.reshape(a,(3,3))
tf.print(tf.reduce_sum(b, axis=1, keepdims=True))
tf.print(tf.reduce_sum(b, axis=0, keepdims=True))

result:

[[6]
 [15]
 [24]]
[[12 15 18]]

bool type reduce

p = tf.constant([True,False,False])
q = tf.constant([False,False,True])
tf.print(tf.reduce_all(p))     ==>0
tf.print(tf.reduce_any(q))     ==>1

2.2.cum scan accumulation

cum scan accumulation:

a = tf.range(1,10)
tf.print(tf.math.cumsum(a))   ==========>[1 3 6 ... 28 36 45]
tf.print(tf.math.cumprod(a))   =========>[1 2 6 ... 5040 40320 362880]

2.3.arg max min index

a = tf.range(1,10)
tf.print(tf.argmax(a))   ==>8
tf.print(tf.argmin(a))   ==>0

2.4.math.top_k ordering tensor

tf.math.top_k can be used to sort tensors

a = tf.constant([1,3,7,5,4,8])

values,indices = tf.math.top_k(a,3,sorted=True)
tf.print(values)
tf.print(indices)

3. Matrix operation

The matrix must be two-dimensional. Such as tf.constant([1,2,3]) is not a matrix.
Matrix operations include: matrix multiplication, matrix transpose, matrix inverse, matrix trace, matrix norm, matrix determinant, matrix eigenvalue, matrix decomposition, etc.
In addition to some common operations, most matrix related operations are in the tf.linalg sub package.

#Matrix multiplication
a = tf.constant([[1,2],[3,4]])
b = tf.constant([[2,0],[0,2]])
a@b  #Equivalent to tf.matmul(a,b)
#Matrix transpose
tf.transpose(a)
#Matrix inverse, must be of type tf.float32 or tf.double
tf.linalg.inv(a)
#Matrix trace
tf.linalg.trace(a)
#Matrix norm
tf.linalg.norm(a)
#Matrix determinant
tf.linalg.det(a)
#Matrix eigenvalue
tf.linalg.eigvals(a)
#Matrix QR decomposition decomposes a square matrix into an orthogonal matrix q and an upper triangular matrix r
#QR decomposition is actually Schmidt orthogonalization of matrix a to obtain q
q,r = tf.linalg.qr(a)
#Matrix svd decomposition
#svd decomposition can decompose any matrix into the product of an orthogonal matrix u, a diagonal matrix s and an orthogonal matrix v.t()
#svd is often used in matrix compression and dimensionality reduction
s,u,v = tf.linalg.svd(a)

4. Broadcasting mechanism tf.broadcast_to(a,b.shape)

Broadcasting is also called broadcasting mechanism (automatic expansion may be more appropriate). It is a lightweight tensor replication means to logically expand the shape of tensor data, but the actual storage replication operation will be performed only when necessary.

If the dimensions of tensors are different, expand the tensor with smaller dimension until the dimensions of the two tensors are the same.
If two tensors have the same length in a certain dimension, or one of them has a length of 1 in that dimension, then we say that the two tensors are compatible in that dimension.
If the two tensors are compatible in all dimensions, they can use broadcasting.
After broadcasting, the length of each dimension will take the larger value of the length of the two tensors in this dimension.
In any dimension, if the length of one tensor is 1 and the length of the other tensor is greater than 1, it is like copying the first tensor in this dimension.

tf.broadcast_to(a,b.shape)`

Example:

a = tf.constant([1,2,3])
b = tf.constant([[0,0,0],[1,1,1],[2,2,2]])
tf.broadcast_to(a,b.shape)
c = tf.constant([1,2,3])
d = tf.constant([[1],[2],[3]])
c+d #Equivalent to tf.broadcast_to(c,[3,3]) + tf.broadcast_to(d,[3,3])