ESPnet is an end-to-end speech processing toolkit, which mainly focuses on end-to-end speech recognition and end-to-end text to speech. ESPnet usage chainer and pytorch As the main deep learning engine, and follow Kaldi Style data processing, feature extraction / format and formula provide complete settings for speech recognition and other speech processing experiments.

System environment information: Ubuntu 18.04.4 LTS, CUDA Version 10.0.130, CUDNN Version 7.6.3,

1. Install vim

When you click the command "vim: get" and "vim: get", you will not be prompted to install vim:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package vim

Run apt get update, and then execute apt get install - y vim

2. Install Anaconda

bash Anaconda3-5.2.0-Linux-x86_64.sh

Then follow the prompts all the time. It is recommended that the default installation location does not need to configure environment variables

Refresh environment variables

If the environment variable is configured, it will not take effect. It will take effect only after refreshing.

source ~/.bashrc

3. Dependent installation

• cmake3 is used for some extensions

  sudo apt-get install cmake

• sox

  sudo apt-get install sox

   

• libsndfile1-dev

  sudo apt-get install libsndfile1-dev

   

• ffmpeg (not required for installation, but used in some places)

  sudo apt-get install ffmpeg

   

• flac (not required for installation, but used in some places)

  sudo apt-get install flac

4. Install kaldi

git clone https://github.com/kaldi-asr/kaldi.git kaldi --origin upstream
apt-get update && \
   　apt-get install -y --no-install-recommends \
        make \
        automake \
        autoconf \
        bzip2 \
        unzip \
        wget \
        libtool \
        git \
        subversion \
        zlib1g-dev \
        gfortran \
        ca-certificates \
        patch \
	vim && \
    rm -rf /var/lib/apt/lists/*
    cd /kaldi/tools && \
    ./extras/install_mkl.sh && \
    make -j4 && \
    cd /kaldi/src && \
    ./configure --shared --use-cuda && \
    make depend -j4 && \
    make -j4 && \

After ALL done OK is displayed, the installation is not completed. You still need to manually install IRSTLM. You can use extras/install_irstlm

After compilation, enter egs/yesno/s5 for execution/ run. If there are no errors and identification results during sh execution, the installation is successful

You can select temporary files generated during compilation

    find /kaldi  -type f \( -name "*.o" -o -name "*.la" -o -name "*.a" \) -exec rm {} \; && \
    find /opt/intel -type f -name "*.a" -exec rm {} \; && \
    find /opt/intel -type f -regex '.*\(_mc.?\|_mic\|_thread\|_ilp64\)\.so' -exec rm {} \; ;&& \
    rm -rf /kaldi/.git

5. Install pytorch

conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.0 -c pytorch

Torch and torch vision are downloaded very slowly and can be downloaded from https://download.pytorch.org/whl/torch_stable.html Download the correct version and install offline with pip instruction

I downloaded torchvision-0.4.0-cp36-cp36m-manylinux1_x86_64.whl and torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl．

6. Install the espent

git clone https://github.com/espnet/espnet
cd espnet
cd tools

To compile and install, you need to specify the installation path of kaldi

#Link Kaldi:
ln -s <kaldi-root> .
#Setting up cuda environment
#Check cuda's path:
whereis cuda
#Then point to the path:
sh ./setup_cuda_env.sh  (Your path)
#Set up Python environment
#I already have Python, but I also installed anaconda for ease of use
#Here I'll add some basic operations of conda command
#Create virtual environment
conda create -n ASR python=3.6
#Activate virtual environment
source activate ASR
#Exit virtual environment
source deactivate
#Reuse and enter the environment
source activate xxxx 
#View existing virtual environments
conda-env list
#When entering the virtual environment, install the corresponding package directly
pip install xxxx perhaps conda install xxxx
#Delete virtual environment: (caution)
conda remove -n xxx --all

#Specifically, I chose scheme a
ln -s <kaldi-root> .
conda create -n espnet python=3.6
conda activate espnet
CONDA_TOOLS_DIR=$(dirname ${CONDA_EXE})/..
./setup_anaconda.sh ${CONDA_TOOLS_DIR} espnet 3.6

make TH_VERSION=1.4.0 

During the installation process, many library files need to be downloaded. The download is very slow, and errors are reported when it is over.

I'll modify the script file and change the PIP source to Tsinghua. Of course, it can also be changed to Alibaba or other sources:

Open the makefile file at lines 127 and 128:

pytorch.done: activate_python.sh
    . ./activate_python.sh && python3 -m pip install numpy --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple some-package  # Workaround for "ImportError: numpy.core.multiarray failed to import"
    . ./activate_python.sh && python3 -m pip install  $(PIP_PYTORCH) -i https://pypi.tuna.tsinghua.edu.cn/simple some-package
    touch pytorch.done

At line 138:

espnet.done: pytorch.done conda_packages.done
    . ./activate_python.sh && python3 -m pip install -e .. -i https://pypi.tuna.tsinghua.edu.cn/simple some-package # Install editable mode by default
    touch espnet.done

At lines 143-150:

ifneq ($(strip $(NO_CUPY)),)
    . ./activate_python.sh && python3 -m pip install $(PIP_CHAINER) -i https://pypi.tuna.tsinghua.edu.cn/simple some-package
else
    # Precompiled cupy==6.0.0 for python>=3.8 is not provided
    . ./activate_python.sh && \
        if python3 -c "import sys; from distutils.version import LooseVersion as L; assert L(sys.version) < L('3.8')" 2>&1 /dev/null; then \
            python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple some-package $(PIP_CHAINER) cupy-cuda$(CUDA_VERSION_WITHOUT_DOT)==$(CHAINER_VERSION) ; \
        else \
            python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple some-package $(PIP_CHAINER) cupy==$(CHAINER_VERSION); \
        fi

Some packages that are only used for specific tasks are not installed by default, so if you encounter some installation errors when running these recipes, you need to choose to install them.  

#Install Warp CTC
. activate_python.sh
. ./setup_cuda_env.sh /usr/local/cuda
./installers/install_warp-ctc.sh

#Install warp transformer
. activate_python.sh
. ./setup_cuda_env.sh /usr/local/cuda
./installers/install_warp-transducer.sh

#Install PyOpenJTalk
. activate_python.sh
./installers/install_pyopenjtalk.sh
 Run error
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 2186: ordinal not in range(128)
It is preliminarily considered to be the problem of Chinese garbled code
  locale
  locale -a
  vim ~/.bashrc
  add to LANG=C.UTF-8  (Some are zh_CN.UTF-8，But I didn't find this code locally)
  source ~/.bashrc
 The problem of Chinese garbled code is solved

#Error after restart
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
#The solution is to run first:
source deactivate
 Rerun
conda deactivate
 Run again conda activate espnet get into
 Re execute the installation command

#Installing ipython using pip
. activate_python.sh
pip install ipython

You can check that the installation completed successfully by

. ./activate_python.sh
python3 check_install.py

In case of the following problems that need to be selected for installation, the installation method is the same as above

INFO:
Use 'installers/install_warp-rnnt.sh' to install warp_rnnt
Use 'installers/install_chainer_ctc.sh' to install chainer_ctc
Use 'installers/install_kenlm.sh' to install kenlm
Use 'installers/install_py3mmseg.sh' to install mmseg
Use 'installers/install_fairseq.sh' to install fairseq
Use 'installers/install_phonemizer.sh' to install phonemizer
Use 'installers/install_gtn.sh' to install gtn
Use 'installers/install_sctk.sh' to install sclite
Use 'installers/install_sph2pipe.sh' to install sph2pipe
Use 'installers/install_pesq.sh' to install PESQ
Use 'installers/install_beamformit.sh' to install BeamformIt
 stay./installers/install_phonemizer.sh This step is wrong,/usr/bin/ld: cannot find -lcurses，terms of settlement, apt-get install libncurses5-dev

Confirm whether the installation is missing

make check_install

Test espnet

Without network, from openslr download wave_yesno Data set and extract to/espnet/egs/yesno/asr1 Folder, and then run bash run.sh，If there is a network, run it directly bash run.sh that will do．

Test whether the GPU of pytorch is available

import torch
flag = torch.cuda.is_available()
print(flag)

ngpu= 1
# Decide which device we want to run on
device = torch.device("cuda:0" if (torch.cuda.is_available() and ngpu > 0) else "cpu")
print(device)
print(torch.cuda.get_device_name(0))
print(torch.rand(3,3).cuda()) 


True
cuda:0
GeForce GTX 1080
tensor([[0.9530, 0.4746, 0.9819],
        [0.7192, 0.9427, 0.6768],
        [0.8594, 0.9490, 0.6551]], device='cuda:0')