For numbers, transform them into matrix form and extract matrix information

It is divided into small blocks. Each block represents whether the position of the number is 1 or 0. The matrix information is extracted through the self coding network

[6538]
[[0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]


[[0. 1. 0. 1.]
 [0. 0. 0. 1.]
 [1. 0. 0. 1.]
 [1. 0. 0. 0.]]

The effect is very good.

step loss  24600 0.00016420467
[60497]
[[1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]
[[9.9977189e-01 1.4487827e-04 6.4183529e-05 2.0390470e-04]
 [9.9998355e-01 1.4964588e-05 9.9996483e-01 5.8404607e-05]
 [4.1264742e-05 1.2945710e-05 9.9998176e-01 9.9994040e-01]
 [7.9966514e-05 9.9992597e-01 9.9983966e-01 9.9984539e-01]]
[[1. 0. 0. 0.]
 [1. 0. 1. 0.]
 [0. 0. 1. 1.]
 [0. 1. 1. 1.]]

The relu activation function can also converge. The key numbers are uncertain each time, and the convergence speed is slightly slow

step loss  2600 0.08281197
[34910]
[[0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]]
[[0.        3.8466296 3.905118  3.8566058]
 [3.8711603 0.        3.889047  0.       ]
 [0.        0.        0.        3.86102  ]
 [0.        0.        0.        3.8216314]]
[[0. 4. 4. 4.]
 [4. 0. 4. 0.]
 [0. 0. 0. 4.]
 [0. 0. 0. 4.]]
step loss  2800 0.057873894








step loss  10600 4.6454425e-05
[48601]
[[1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1.]]
[[3.159582  0.        0.        3.1603072]
 [3.1596205 0.        3.1602912 3.160251 ]
 [3.16022   0.        3.1602015 3.1602106]
 [3.1603267 3.1603467 0.        3.1603827]]
[[3. 0. 0. 3.]
 [3. 0. 3. 3.]
 [3. 0. 3. 3.]
 [3. 3. 0. 3.]]

tanh, because of the mapping space [- 1,1], there will be a negative number

step loss  11600 0.010202475
[38303]
[[1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 1. 1. 1. 1. 0. 0. 0. 0.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]
 [1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 1. 1.]]
[[ 0.99622285  0.9966955   0.9966803   0.99454   ]
 [ 0.9975496  -0.9544199  -0.954272    0.99737585]
 [ 0.99751127 -0.9543707   0.9984945  -0.9541669 ]
 [ 0.99677926 -0.9542868  -0.9543052   0.99700284]]
[[ 1.  1.  1.  1.]
 [ 1. -1. -1.  1.]
 [ 1. -1.  1. -1.]
 [ 1. -1. -1.  1.]]

It can be seen that for specific problems, it is very important to select the appropriate activation function according to the needs

import numpy as np
import tensorflow as tf
import tensorflow.contrib.slim as slim

box_size = 4  # Each 1 represents a 3 represents a 3 x 3 represents a 1 block
box_num = 4  # The number of lattices is 4, indicating that the matrix is divided into 4X4 lattices, and each lattice represents a bit
mat_size = box_num * box_size
train_steps = 10000
show_steps = 200
batch_size = 16
learning_rate = .001


# Digital transformation matrix
# (Num, 3, 4) return 1 x 12 x 12 x 1
def num2mat(num, box_size, box_num):
    rows = []
    for i in range(box_num):
        row = []
        for j in range(box_num):
            # Is the bit 0 or 1
            n = (num >> (i * box_num + j)) & 1
            row.append(np.full(shape=(box_size, box_size, 1), fill_value=n, dtype=np.float32))
        row = np.concatenate(row, axis=1)
        rows.append(row)

    mat = np.concatenate(rows, axis=0)
    return mat


def get_data(batch_size, box_size, box_num):
    max_num = 1 << box_num ** 2
    y_data = np.random.randint(0, max_num, [batch_size, 1])
    x_data = np.stack([num2mat(i, box_size, box_num) for i in y_data])
    return x_data, y_data


x_data, y_data = get_data(batch_size, box_size, box_num)
print(x_data.shape, y_data.shape)  # (16, 16, 16, 1) (16, 1)
# print(x_data[0].reshape(9, 9), y_data[0])

in_x = tf.placeholder(tf.float32, (None, mat_size, mat_size, 1))
print(in_x.shape)  # (?, 16, 16, 1)

with slim.arg_scope(
        [slim.conv2d_transpose, slim.conv2d],
        # activation_fn=tf.nn.sigmoid,
        # activation_fn=tf.nn.relu,
        activation_fn=tf.nn.tanh,
):
    net = slim.conv2d(in_x, 3, box_size, 2)
    print(net.shape)  # (?, 8, 8, 3)
    out_mat = slim.conv2d(net, 1, box_size, 2)
    print(out_mat.shape)  # (?, 4, 4, 1)
    net = slim.conv2d_transpose(out_mat, 3, box_size, 2)
    print(net.shape)  # (?, 8, 8, 3)
    out_y = slim.conv2d_transpose(net, 1, box_size, 2)
    print(out_y.shape)  # (?, 16, 16, 1)

loss = tf.reduce_sum((out_y - in_x) ** 2)
train_op = tf.train.AdamOptimizer(learning_rate).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for i in range(train_steps):
        x_data, y_data = get_data(batch_size, box_size, box_num)
        sess.run(train_op, feed_dict={
            in_x: x_data
        })

        if not (i + 1) % show_steps:
            x_data, y_data = get_data(batch_size, box_size, box_num)
            out_mat_val, out_y_val, loss_val = sess.run(
                [out_mat, out_y, loss],
                feed_dict={
                    in_x: x_data
                })

            print('step loss ', i + 1, loss_val)
            print(y_data[0])
            print(x_data[0].reshape(mat_size, mat_size))
            print(out_mat_val[0].reshape(box_size, box_size))
            print(np.rint(out_mat_val[0]).reshape(box_size, box_size))