[🖤PaddleGAN Double eleven special 🖤] "Sorry, I fell in love with myself of the opposite sex"

💛 This project is based on [Aimeng] boss project Adaptation 🧡

Thanks again for AI Meng's contribution (◍ ᴗ ◍) 💛， We also welcome more partners to join us PaddleGAN • SIG team to build with us ̀ ᴗ• ́)و ̑̑~

Double eleven is coming again. No, no, you're still single Д ﾟ=)

No problem! PaddleGAN On this important day, let's solve everyone's problems again!

No object is a big deal, PaddleGAN The gender conversion function makes you fall in love with yourself( ˘ ³˘) ♥

Which uses PaddleGAN The face attribute editing ability in realizes the gender transformation of any face!

At the same time, the ability of facial movement transfer makes you "move" of the opposite sex more realistic!

All the code and detailed tutorials are in PaddleGAN Welcome, Star ⭐ Collect all dry goods!

📖 Uncover the secret of principle

Basic knowledge

📌 Late Code: potential factor. Each image corresponds to a potential factor (high-dimensional vector). stylegan can use this vector to generate images.

📌 Attribute editing: because potential factors contain many dimensions, it is impossible to determine the direction corresponding to the dimension and the changes caused by editing. Therefore, a large number of potential factors and corresponding attributes (such as age or gender) are used to train a linear classifier, and the weight of the classifier can be used as the direction of potential factors and corresponding attributes

📌 StyleGAN: generate pictures based on vectors

Principle steps

There are three steps to realize face fusion:

1. Obtain the late code of the image for subsequent attribute editing and face generation
2. Edit the Latent Code according to a specific direction to edit the corresponding face attributes, such as age, gender, hair, eyes, etc
3. StyleGAN V2 generates the target face according to the Latent Code vector edited in step 2

⏳ Pre step: installation PaddleGAN And related dependencies

In this project, we have completed the latest version of paddegan for git clone. There is no need to re clone, but ppgan needs to be installed. Install it in sequence~

# clone the paddegan code from github (if the download speed is too slow, the gitee source can be used). The project has been cloned, and this step can be omitted
#!git clone https://github.com/PaddlePaddle/PaddleGAN
#!git clone https://gitee.com/PaddlePaddle/PaddleGAN

# Installation dependency
%cd /home/aistudio/PaddleGAN
!pip install -r requirements.txt

#Install paddegan locally
!python setup.py develop 

#Install the required package. The first installation of this package will take approximately 5 minutes
!pip install dlib 

STEP 1: obtain the Latent Code corresponding to the original drawing

Here, we use Pixel2Style2Pixel to extract late code. You only need to change two parameters:

1. input_image: original image path, that is, the photo path that needs to extract hidden features
2. output_path: the storage path of the hidden features of the original drawing, which needs to be used in the attribute editing and generation module later

Put the input_image part on the face photos that you want to change gender. Please note that it is best to take self photos and large head photos. It is better to have no glasses~

%cd applications/
!python -u tools/pixel2style2pixel.py \
       --input_image '/home/aistudio/Face test set/yingbb2.png' \
       --output_path   '/home/aistudio/psp_output/yingbb2' \
       --model_type ffhq-inversion \
       --seed 233 \
       --size 1024 \
       --style_dim 512 \
       --n_mlp 8 \
       --channel_multiplier 2 

Step 2 & 3: edit the Latent Code in a specific direction, and use StyleGAN V2 to generate a new face from the edited vector

You only need to change three parameters:

1. Later: late code of the original drawing extracted in STEP2 (output_path path in STEP2)
2. output_path: the saving path of the new face (after gender transformation)

Original drawing	After gender change	Dynamic effect

!python -u tools/styleganv2editing.py \
       --latent '/home/aistudio/psp_output/11/dst.npy' \
       --output_path '/home/aistudio/final_output/11' \
       --model_type ffhq-config-f \
       --size 1024 \
       --style_dim 512 \
       --n_mlp 8 \
       --channel_multiplier 2 \
       --direction_name gender \
       --direction_offset 5

💟 BONUS experience: let you of the opposite sex "move" and really fall in love with yourself 💟

the latest version PaddleGAN Medium FOM The model supports the generation of 512 definition video, the definition is doubled, and the function of face enhancement is added to depict rich facial details, making the dynamic facial expression more vivid~

Among them, the principle of FOM is to move the facial action of character A to the face of character B, so that the face of character B can perfectly interpret the expression of character A.

By modifying the two parameters of driving_video and source_image in the following command, replace them with your own driving video (the facial expression you want the opposite sex to make to you) and your photos of the opposite sex, you can see your vivid other side~

The final video export is saved in the "dynamic version of the opposite sex" folder~

Come and try 😎

%cd applications
!python -u tools/first-order-demo.py  \
     --driving_video '/home/aistudio/Drive video.MOV' \
     --source_image '/home/aistudio/final_output/11_5/dst.editing.png' \
     --relative \
     --adapt_scale \
     --output '/home/aistudio/Heterosexual dynamic version' \
     --image_size 512 \
     --face_enhancement

PaddleGAN There are many other abilities in, you can experience~

• "Face fusion"

Character 1	Character 2	Effect after fusion

• "Girl with Pearl Earrings" broadcast the news

• Centennial image coloring, frame interpolation and improved resolution

• Photo Animation

• Face animation

⭐ ⭐ ⭐ welcome Star support! ⭐ ⭐ ⭐

⭐ Finally, special thanks for contributing this ability to PaddleGAN [AI Meng] boss in ⭐， Original project link: https://aistudio.baidu.com/aistudio/projectdetail/2377580?channelType=0&channel=0