Old Photo Restoration (Paddle Implementation)
github link: https://github.com/buriedms/Old2Life-Paddle.git
The reproduced paper is: Old Photo Restoration via Deep Latent Space Translation
Official open source pytorch Code: Bringing-Old-Photos-Back-to-Life
The data set adopts VOC data set, and a small number of real old photos and old photos Data address
Model and configuration file storage Network disk link , extraction code: xfmp
Visual rendering Network disk link , extraction code: s32p
Training data Network disk link , extraction code: 07oi
Recurrence index
| Performance | PSNR | SSIM | FID | LPIPS |
|---|---|---|---|---|
| Target | 23.33 | 0.69 | 134.35 | 0.25 |
| stage_A/Epoch(20) | 23.929 | 0.749 | 31.928 | 0.302 |
| stage_B/Epoch(20) | 24.269 | 0.834 | 21.873 | 0.189 |
| stage_Map/Epoch(20/A:20,B:20) | 22.930 | 0.709 | 122.859 | 0.321 |
abstract
We propose a deep learning method to restore seriously degraded old photos. Different from the traditional restoration tasks that can be solved by supervised learning, the degradation of real photos is very complex, and the domain gap between synthetic images and real old photos makes the network unable to be generalized. Therefore, we propose a new triple domain translation network using real photos and a large number of synthetic image pairs.
Specifically, we train two variational automatic encoders (VAEs) to convert old photos and clean photos into two potential spaces respectively. The transformation between these two potential spaces is learned using synthetic paired data. This translation can be well extended to real photos because the domain gap is closed in a compact potential space. In addition, in order to solve the multiple degradation problems mixed in an old photo, we designed a global branch, designed a local nonlocal block for structured defects such as scratches and dust spots, and designed a local branch for unstructured defects such as noise and blur. The two branches fuse in potential space, which improves the ability to recover old photos from multiple defects. In addition, we use another face thinning network to restore the fine details of the face in the old photos, and finally generate the photos with enhanced perceptual quality. Through comprehensive experiments, the proposed pipeline proves to be superior to the most advanced methods and existing commercial tools in the visual quality of old photo restoration.
Figure 1: old photo restoration results generated by our method. This method can deal with the complex degradation problem mixed with unstructured and structured defects in real old photos. In particular, we recover the high-frequency details of the face region and further improve the perceptual quality of the portrait. For each image pair, the input is on the left and the modified output is displayed on the right.
1, Preparation before experiment
Data set and pre training model
Training data
- False old photos were synthesized using images from Pascal VOC dataset.
- Collected real image old photo data sets - color and black and white.
Data sets can be obtained in two ways:
- adopt Network disk link Download datasets,
Extraction code: 07oi.
Note: it has been publicly produced on aistudio data set - adopt Custom data file Create. Production details can be viewed and learned through old2life pad / global / data / readme.md
Pre training model
Twenty epoch s have been provided for each phase of training Training model , it can be used for testing.
Operating environment
| Environment | version |
|---|---|
| Paddle with Tesla V-100 | 2.1.2 |
Install dependent Libraries
!pip install x2paddle
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting x2paddle Using cached https://pypi.tuna.tsinghua.edu.cn/packages/c1/82/49ac2659d7598ddb055316aca92b4cf24c2c3c5df9e1a3cdf7dce3c84f23/x2paddle-1.3.4-py3-none-any.whl Installing collected packages: x2paddle Successfully installed x2paddle-1.3.4
Prepare dataset
Extract the configuration file and data file to the specified folder
!unzip /home/aistudio/data/data114496/old2life.zip -d work/old2life/ #Training data file / home/aistudio/data/data114496/old2life.zip !unzip /home/aistudio/data/data114816/checkpoints.zip -d work/Old2Life/ # Model configuration file / home/aistudio/data/data114816/checkpoints.zip
Archive: /home/aistudio/data/data114496/old2life.zip creating: work/old2life/old2life/ inflating: work/old2life/old2life/Real_L_old.bigfile inflating: work/old2life/old2life/Real_RGB_old.bigfile inflating: work/old2life/old2life/VOC.bigfile Archive: /home/aistudio/data/data114816/checkpoints.zip creating: work/Old2Life/checkpoints/ creating: work/Old2Life/checkpoints/domainA_SR_old_photos/ inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_net_D.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_net_featD.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_net_G.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_optimizer_D.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_optimizer_featD.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/best_optimizer_G.pdparams extracting: work/Old2Life/checkpoints/domainA_SR_old_photos/iter.txt inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_net_D.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_net_featD.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_net_G.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_optimizer_D.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_optimizer_featD.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/latest_optimizer_G.pdparams inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/loss_log.txt inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/model.txt inflating: work/Old2Life/checkpoints/domainA_SR_old_photos/opt.txt creating: work/Old2Life/checkpoints/domainB_old_photos/ inflating: work/Old2Life/checkpoints/domainB_old_photos/best_net_D.pdparams inflating: work/Old2Life/checkpoints/domainB_old_photos/best_net_G.pdparams inflating: work/Old2Life/checkpoints/domainB_old_photos/best_optimizer_D.pdparams inflating: work/Old2Life/checkpoints/domainB_old_photos/best_optimizer_G.pdparams extracting: work/Old2Life/checkpoints/domainB_old_photos/iter.txt inflating: work/Old2Life/checkpoints/domainB_old_photos/latest_net_D.pdparams inflating: work/Old2Life/checkpoints/domainB_old_photos/latest_net_G.pdparams inflating: work/Old2Life/checkpoints/domainB_old_photos/loss_log.txt inflating: work/Old2Life/checkpoints/domainB_old_photos/model.txt creating: work/Old2Life/checkpoints/mapping_quality/ inflating: work/Old2Life/checkpoints/mapping_quality/best_net_mapping_net.pdparams extracting: work/Old2Life/checkpoints/mapping_quality/iter.txt inflating: work/Old2Life/checkpoints/mapping_quality/latest_net_mapping_net.pdparams inflating: work/Old2Life/checkpoints/mapping_quality/loss_log.txt inflating: work/Old2Life/checkpoints/mapping_quality/model.txt inflating: work/Old2Life/checkpoints/mapping_quality/opt.txt
Enter working directory
Note: this step is required to run with Notebook
# Enter the working directory and use Notebook for training. This step is required %cd Old2Life-Paddle/
/home/aistudio/Old2Life-Paddle
2, Training start
1. Terminal startup
- The terminal executes the following commands in the old2life paddle file directory:
bash train.sh
- If you want to train a single stage, you can execute the following command.
a. First stage of training
b. Training phase IIbash Global/run_a.sh
c. The third stage of trainingbash Global/run_b.sh
bash Global/run_map.sh
If you need to change the training parameters, you can modify them in.
2.Notebook startup
-
Execute the following command in the working directory old2life paddle:
bash train.sh
-
If you want to train a single stage, you can execute the following command.
a. First stage of training
!bash Global/run_a.sh
b. Training phase II
!bash Global/run_b.sh
c. The third stage of training
!bash Global/run_map.sh
Required parameter interpretation
| parameter | explain | case |
|---|---|---|
| dataroor | The location where the picture data is stored. The format is. bigfile. | --dataroot /home/aistudio/work/Old2Life/test_old |
| output_dir | Picture output path. | --outputs_dir /home/aistudio/work/Old2Life/output/ |
| checkpoints_dir | Save result parameters and training log storage path. | --checkpoints_dir /home/aistudio/work/Old2Life/test_checkpoints |
| Important optional parameters | - | - |
| batchSize | Number of samples selected for one training | --batchSize 64 |
| gpu_ids | Selected gpu serial number | --gpu_ids 0,1,2,4 |
| use_vae_which_epoch | The version of vae model selected for pre training | --use_vae_which_epoch latest |
| which_epoch | Model version used for pre training | --which_epoch latest |
| niter | The learning rate does not decay the number of training rounds | --niter 15 |
| niter_decay | Number of rounds of learning rate attenuation training | --niter_decay 15 |
| continue_train | Whether to use the pre training model for continuous learning and trigger it to take effect | --continue_train |
| debug | Whether to enable the debug mode and trigger it to take effect | --debug |
# Notebook test code # Start training at all stages !bash train.sh # # Start training in phase A # !bash Global/run_a.sh # # Phase B training start # !bash Global/run_b.sh # # MAP phase training start # !bash Global/run_map.sh
Trian A Start !
CustomDatasetDataLoader
dataset [UnPairOldPhotos_SR] was created
start load bigfile (0.00 GB) into memory
find total 4 images
load 0 images done
load all 4 images done
start load bigfile (0.01 GB) into memory
find total 8 images
load 0 images done
load all 8 images done
start load bigfile (0.09 GB) into memory
find total 966 images
load 0 images done
load all 966 images done
-------------Filter the imgs whose size <256 in VOC-------------
--------Origin image num is [966], filtered result is [941]--------
#training images = 960
12/02/2021 11:46:02 - INFO - root - ================ Training domainA_SR_old_photos Loss (Thu Dec 2 11:46:02 2021) ================
12/02/2021 11:46:02 - INFO - root - Resuming from epoch 3 at iteration 0
W1202 11:46:02.908684 1475 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W1202 11:46:02.913003 1475 device_context.cc:422] device: 0, cuDNN Version: 7.6.
network import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_net_G.pdparams
---------- G Networks reloaded -------------
network import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_net_D.pdparams
network import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_net_featD.pdparams
---------- D Networks reloaded -------------
12/02/2021 11:46:08 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:46:08 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/vgg19-pt.pdparams
---------- Optimizers initialized -------------
optimizer import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_optimizer_D.pdparams
optimizer import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_optimizer_G.pdparams
optimizer import path: /home/aistudio/work/Old2Life/test_checkpoints/domainA_SR_old_photos/latest_optimizer_featD.pdparams
---------- Optimizers reloaded -------------
---------- Current LR is 0.00020000 -------------
12/02/2021 11:46:17 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:46:17 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/InceptionV3.pdparams
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
PerceptualVGG loaded pretrained weight.
12/02/2021 11:46:23 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:46:23 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
Loading model from: /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
12/02/2021 11:46:28 - INFO - root - Epoch: 4, Iters: 64, Time: 0.036 lr: 0.00020 || G_GAN: 0.773 G_GAN_Feat: 9.242 G_VGG: 7.741 G_KL: 1.010 D_real: 0.663 D_fake: 0.648 G_featD: 0.400 featD_real: 0.355 featD_fake: 0.287
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:706: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
elif dtype == np.bool:
12/02/2021 11:46:31 - INFO - root - Epoch: 4, Iters: 128, Time: 0.014 lr: 0.00020 || G_GAN: 0.787 G_GAN_Feat: 9.330 G_VGG: 7.891 G_KL: 1.002 D_real: 0.682 D_fake: 0.655 G_featD: 0.384 featD_real: 0.323 featD_fake: 0.311
12/02/2021 11:46:35 - INFO - root - Epoch: 4, Iters: 192, Time: 0.014 lr: 0.00020 || G_GAN: 0.803 G_GAN_Feat: 9.168 G_VGG: 7.775 G_KL: 0.977 D_real: 0.718 D_fake: 0.651 G_featD: 0.373 featD_real: 0.340 featD_fake: 0.345
12/02/2021 11:46:38 - INFO - root - Epoch: 4, Iters: 256, Time: 0.014 lr: 0.00020 || G_GAN: 0.816 G_GAN_Feat: 10.262 G_VGG: 7.874 G_KL: 0.986 D_real: 0.704 D_fake: 0.679 G_featD: 0.419 featD_real: 0.350 featD_fake: 0.274
^C
Traceback (most recent call last):
File "Global/train_domain_A.py", line 107, in <module>
loss_D.backward()
File "<decorator-gen-122>", line 2, in backward
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
return wrapped_func(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py", line 227, in __impl__
return func(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/varbase_patch_methods.py", line 239, in backward
framework._dygraph_tracer())
KeyboardInterrupt
Train A Over !
Tarin B Start !
CustomDatasetDataLoader
dataset [UnPairOldPhotos_SR] was created
start load bigfile (0.09 GB) into memory
find total 966 images
load 0 images done
load all 966 images done
-------------Filter the imgs whose size <256 in VOC-------------
--------Origin image num is [966], filtered result is [941]--------
#training images = 960
12/02/2021 11:46:46 - INFO - root - ================ Training domainB_old_photos Loss (Thu Dec 2 11:46:46 2021) ================
12/02/2021 11:46:46 - INFO - root - Resuming from epoch 1 at iteration 0
W1202 11:46:46.363127 1628 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W1202 11:46:46.367645 1628 device_context.cc:422] device: 0, cuDNN Version: 7.6.
network import path: /home/aistudio/work/Old2Life/test_checkpoints/domainB_old_photos/latest_net_G.pdparams
---------- G Networks reloaded -------------
network import path: /home/aistudio/work/Old2Life/test_checkpoints/domainB_old_photos/latest_net_D.pdparams
---------- D Networks reloaded -------------
12/02/2021 11:46:51 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:46:51 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/vgg19-pt.pdparams
---------- Optimizers initialized -------------
optimizer import path: /home/aistudio/work/Old2Life/test_checkpoints/domainB_old_photos/latest_optimizer_D.pdparams
optimizer import path: /home/aistudio/work/Old2Life/test_checkpoints/domainB_old_photos/latest_optimizer_G.pdparams
---------- Optimizers reloaded -------------
---------- Current LR is 0.00020000 -------------
12/02/2021 11:47:00 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:47:00 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/InceptionV3.pdparams
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
PerceptualVGG loaded pretrained weight.
12/02/2021 11:47:07 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:47:07 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
Loading model from: /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
12/02/2021 11:47:10 - INFO - root - Epoch: 2, Iters: 64, Time: 0.034 lr: 0.00020 || G_GAN: 0.759 G_GAN_Feat: 10.868 G_VGG: 10.703 G_KL: 2.009 D_real: 0.757 D_fake: 0.760
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:706: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
elif dtype == np.bool:
12/02/2021 11:47:13 - INFO - root - Epoch: 2, Iters: 128, Time: 0.011 lr: 0.00020 || G_GAN: 0.758 G_GAN_Feat: 11.050 G_VGG: 11.088 G_KL: 1.911 D_real: 0.779 D_fake: 0.710
12/02/2021 11:47:16 - INFO - root - Epoch: 2, Iters: 192, Time: 0.011 lr: 0.00020 || G_GAN: 0.770 G_GAN_Feat: 10.839 G_VGG: 10.811 G_KL: 1.864 D_real: 0.789 D_fake: 0.683
12/02/2021 11:47:19 - INFO - root - Epoch: 2, Iters: 256, Time: 0.011 lr: 0.00020 || G_GAN: 0.768 G_GAN_Feat: 11.603 G_VGG: 10.854 G_KL: 1.830 D_real: 0.786 D_fake: 0.695
^C
Traceback (most recent call last):
File "Global/train_domain_B.py", line 144, in <module>
performance.update(generated[:5], data['image'][:5]) if dist.get_rank()==0 else None
File "/home/aistudio/Old2Life-Paddle/Global/util/util.py", line 64, in update
self.FID.update(preds,gts)
File "/home/aistudio/Old2Life-Paddle/Global/util/FID.py", line 69, in update
preds, gts, self.batch_size, self.model, self.use_GPU, self.dims)
File "/home/aistudio/Old2Life-Paddle/Global/util/FID.py", line 153, in calculate_inception_val
use_gpu)
File "/home/aistudio/Old2Life-Paddle/Global/util/FID.py", line 133, in _get_activations_from_ims
pred = model(images)[0][0]
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__
outputs = self.forward(*inputs, **kwargs)
File "/home/aistudio/Old2Life-Paddle/Global/util/inception.py", line 129, in forward
x = block(x)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__
outputs = self.forward(*inputs, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/container.py", line 98, in forward
input = layer(input)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__
outputs = self.forward(*inputs, **kwargs)
File "/home/aistudio/Old2Life-Paddle/Global/util/inception.py", line 641, in forward
branch_pool = self.branch_pool(branch_pool)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__
outputs = self.forward(*inputs, **kwargs)
File "/home/aistudio/Old2Life-Paddle/Global/util/inception.py", line 746, in forward
y = self.bn(y)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__
outputs = self.forward(*inputs, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/nn.py", line 1342, in forward
mean_out, variance_out, *attrs)
KeyboardInterrupt
Train B Over!
Train MAP Start !
12/02/2021 11:47:23 - INFO - root - ================ Training mapping_quality Loss (Thu Dec 2 11:47:23 2021) ================
12/02/2021 11:47:23 - INFO - root - Resuming from epoch 1 at iteration 0
CustomDatasetDataLoader
dataset [PairOldPhotos] was created
start load bigfile (0.09 GB) into memory
find total 966 images
load 0 images done
load all 966 images done
-------------Filter the imgs whose size <256 in VOC-------------
--------Origin image num is [966], filtered result is [941]--------
#training images = 928
W1202 11:47:26.831796 1680 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W1202 11:47:26.836323 1680 device_context.cc:422] device: 0, cuDNN Version: 7.6.
Mapping: You are using the mapping model without global restoration.
network import path: /home/aistudio/work/Old2Life/checkpoints/domainA_SR_old_photos/latest_net_G.pdparams
network import path: /home/aistudio/work/Old2Life/checkpoints/domainB_old_photos/latest_net_G.pdparams
network import path: /home/aistudio/work/Old2Life/test_checkpoints/mapping_quality/latest_net_mapping_net.pdparams
L1Loss()
12/02/2021 11:47:32 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:47:32 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/vgg19-pt.pdparams
---------- Optimizers initialized -------------
12/02/2021 11:47:41 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:47:41 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/InceptionV3.pdparams
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/creation.py:125: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
if data.dtype == np.object:
PerceptualVGG loaded pretrained weight.
12/02/2021 11:47:48 - INFO - paddle.utils.download - unique_endpoints {''}
12/02/2021 11:47:48 - INFO - paddle.utils.download - Found /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
Loading model from: /home/aistudio/.cache/paddle/hapi/weights/lins_0.1_vgg.pdparams
12/02/2021 11:47:53 - INFO - root - Epoch: 2, Iters: 32, Time: 0.070 lr: 0.00020 || G_Feat_L2: 47.417 G_GAN: 2.736 G_GAN_Feat: 8.594 G_VGG: 10.082 D_real: 2.681 D_fake: 2.012
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py:706: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
elif dtype == np.bool:
12/02/2021 11:47:57 - INFO - root - Epoch: 2, Iters: 64, Time: 0.031 lr: 0.00020 || G_Feat_L2: 48.239 G_GAN: 45.647 G_GAN_Feat: 8.751 G_VGG: 10.735 D_real: 41.937 D_fake: 46.040
12/02/2021 11:48:01 - INFO - root - Epoch: 2, Iters: 96, Time: 0.031 lr: 0.00020 || G_Feat_L2: 50.760 G_GAN: 8.463 G_GAN_Feat: 11.348 G_VGG: 11.035 D_real: 11.230 D_fake: 8.833
12/02/2021 11:48:05 - INFO - root - Epoch: 2, Iters: 128, Time: 0.031 lr: 0.00020 || G_Feat_L2: 48.898 G_GAN: 13.146 G_GAN_Feat: 10.431 G_VGG: 10.061 D_real: 13.486 D_fake: 13.112
^C
Traceback (most recent call last):
File "Global/train_mapping.py", line 132, in <module>
loss_D.backward()
File "<decorator-gen-122>", line 2, in backward
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
return wrapped_func(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/framework.py", line 227, in __impl__
return func(*args, **kwargs)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/varbase_patch_methods.py", line 239, in backward
framework._dygraph_tracer())
KeyboardInterrupt
Train MAP Over !
3, Test evaluation
1. Terminal test
View test indicators
The terminal executes the following commands in the old2life paddle Directory:
bash test_Sea.sh
View picture reconstruction visualization
The terminal executes the following commands in the old2life paddle Directory:
bash test_Elm.sh
2.Notebook test
!bash test_Sea.sh !bash test_Elm.sh
Required parameter interpretation
| parameter | explain | case |
|---|---|---|
| load_pretrainA | The path folder where the phase A training model is stored. | --dataroot /home/aistudio/work/Old2Life/test_old |
| load_pretrainB | The path folder where the phase B training model is stored. | --outputs_dir /home/aistudio/work/Old2Life/output/ |
| dataroot | Test the storage path of the picture of the performance index. | --checkpoints_dir /home/aistudio/work/Old2Life/test_checkpoints |
| checkpoints_dir | The location of the master file where the configuration information and model information are stored. | --checkpoints_dir /home/aistudio/work/Old2Life/test_checkpoints |
| test_input | Test the location of old photos. | --checkpoints_dir /home/aistudio/work/Old2Life/test_checkpoints |
| output_dir | Converted picture output path. | --checkpoints_dir /home/aistudio/work/Old2Life/test_checkpoints |
| Important optional parameters | - | - |
| batchSize | Number of samples selected for the test | --batchSize 8 |
| gpu_ids | Selected gpu serial number | --gpu_ids 0 |
| which_epoch | Model version used for testing | --which_epoch latest |
# Notebook test code # Obtain the test index results of the image !bash test_Sea.sh # Obtain the generation effect of image reconstruction !bash test_Elm.sh
4, Finished product effect display
Figure 2: the original test picture is at the top and the reconstructed picture is at the bottom.
It is obvious that the image reconstructed by the model is more satisfactory in terms of clarity and color saturation,
This effect is the result of 20 epoch s trained by the model. Later, with the training index still rising slowly, the result in the original paper is
With the results of 200 epoch s, we have reason to believe that the effect we show will not be the best effect with the number of training rounds
The reconstruction effect can still be improved.