java implementation of SFTP and SSH

Posted by Voodoo Jai on Thu, 13 Jan 2022 12:45:58 +0100

sftp and ftp

To talk about sftp (SSH File Transfer Protocol), first talk about ftp (File Transfer Protocol). As we all know, ftp is a file transfer protocol. It is based on tcp protocol and can be used to send files.

sftp and ssh

sftp is the ftp of security, because it is based on SSH protocol. SSH is the abbreviation of Secure Shell, which is formulated by the Network Working Group of IETF; SSH is a security protocol based on the application layer. SSH is a reliable protocol designed to provide security for remote login sessions and other network services. Using SSH protocol can effectively prevent information leakage in the process of remote management. SSH will encrypt all data in the process of connection and transmission. Therefore, generally speaking, file transfer through SSH protocol is sftp.

So how to use ssh to realize file transfer? Friends who are familiar with linux should be familiar with ssh, because linux comes with ssh (Shh! In fact, I'm not familiar with linux, but I know that linux comes with ssh, which was installed before long, but now I find that linux will improve a lot. I must install another one to play!).

Unfortunately, ssh is basically based on linux and some client installation software. So what if we use sftp to transfer files in our ordinary web development? jsch is the solution.

Introduction to jsch

jsch is a pure java implementation of ssh. It's a little abstract. Generally speaking, what are you doing On the official website down is a jar package. Introduce your project and you can start using it

Step 1: first, check how to rely on pom in maven central warehouse. You can click here.

 <!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
 <dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.55</version>
</dependency>

Step 2: create a tool class: sftpclient Java to realize file upload, download, read and other functions: purchasing on behalf of others is as follows:

package com.zjh.logviewer.ssh;

import com.zjh.logviewer.model.FileAttri;
import com.zjh.logviewer.model.Server;


import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Vector;

import org.apache.commons.lang3.ArrayUtils;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.ChannelSftp.LsEntry;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.SftpATTRS;
import com.jcraft.jsch.SftpException;

public class SFTPClient extends BaseJchClient {

    private ChannelSftp channel;

    public SFTPClient(Server serverInfo) throws RemoteAccessException {
        super(serverInfo);
    }

    private ChannelSftp openChannel() throws RemoteAccessException {
        try {
            if (channel != null) {
                if (channel.isConnected() || !channel.isClosed()) {
                    channel.disconnect();
                    channel = null;
                } else {
                    return channel;
                }
            }
            conn();

            channel = (ChannelSftp) session.openChannel("sftp");
            channel.connect(DEFAULT_CONN_TIMEOUT);
            return channel;

        } catch (JSchException e) {
            throw new RemoteAccessException(e);
        }
    }

    /**
     * Download the specified file from the sftp server to the specified local directory
     * 
     * @param remoteFile Absolute path of file + fileName
     * @param localPath Local temporary file path
     * @return
     */
    public boolean download(String remoteFile, String localPath) throws RemoteAccessException {
        ChannelSftp sftp = null;
        try {
            sftp = openChannel();
            sftp.get(remoteFile, localPath);
            return true;
        } catch (SftpException e) {
            logger.error("download remoteFile:{},localPath:{}, ex:{}", remoteFile, localPath, e);
            throw new RemoteAccessException(e);
        }
    }

    /**
     * Upload file
     *
     * @param directory  Uploaded Directory - relative to the user access directory set by SFPT, if it is empty, a file will be created in the root directory set by SFTP (except that server full disk access is set)
     * @param uploadFile Full path of file to upload
     */
    public  boolean upload(String directory, String uploadFile) throws Exception {
        ChannelSftp sftp = null;
        try {
            try {
                sftp = openChannel();
                sftp.cd(directory); // Enter directory
            } catch (SftpException sException) {
                if (sftp.SSH_FX_NO_SUCH_FILE == sException.id) { // The specified upload path does not exist
                    sftp.mkdir(directory);// Create directory
                    sftp.cd(directory); // Enter directory
                }
            }

            File file = new File(uploadFile);
            InputStream in = new FileInputStream(file);

            sftp.put(in, file.getName());
            in.close();
        } catch (Exception e) {
            throw new Exception(e.getMessage(), e);
        }
        return true;
    }

    /**
     * Read the specified file data on sftp
     * 
     * @param remoteFile
     * @return
     */
    public byte[] getFile(String remoteFile) throws RemoteAccessException {
        ChannelSftp sftp = null;
        InputStream inputStream = null;
        try {
            sftp = openChannel();
            inputStream = sftp.get(remoteFile);
            return IOHelper.readBytes(inputStream);
        } catch (SftpException | IOException e) {
            logger.error("getFile remoteFile:{},ex:{}", remoteFile, e);
            throw new RemoteAccessException(e);
        } finally {
            IOHelper.closeQuietly(inputStream);
        }
    }

    /**
     * Read the specified (text) file data on sftp and return the data set by line
     *
     * @param remoteFile
     * @param charset
     * @return
     */
    public String getFileContent(String remoteFile, Charset charset) throws RemoteAccessException {
        ChannelSftp sftp = null;
        InputStream inputStream = null;
        try {
            sftp = openChannel();
            inputStream = sftp.get(remoteFile);
            return IOHelper.readText(inputStream, charset);
        } catch (SftpException | IOException e) {
            logger.error("getFileText remoteFile:{},error:{}", remoteFile, e);
            throw new RemoteAccessException(e);
        } finally {
            IOHelper.closeQuietly(inputStream);
        }
    }

    /**
     * Lists the files in the specified directory
     * 
     * @param remotePath
     * @param descendant Recursively query descendant directory
     * @param excludes Files to exclude
     * @return
     */
    public List<FileAttri> ls(String remotePath, boolean descendant, String... excludes) throws RemoteAccessException {
        ChannelSftp sftp = null;
        List<FileAttri> lsFiles;
        try {
            sftp = openChannel();
            lsFiles = ls(sftp, remotePath, descendant, excludes);
        } catch (SftpException e) {
            logger.error("ls remotePath:{} , error:{}", remotePath, e.getMessage());
            if ("Permission denied".equals(e.getMessage())) {
                throw new PermissionException("No file read permission");
            }
            throw new RemoteAccessException(e);
        }
        if (lsFiles != null) {
            Collections.sort(lsFiles);
        }
        return lsFiles;
    }

    @SuppressWarnings("unchecked")
    private List<FileAttri> ls(ChannelSftp sftp, String remotePath, boolean descendant, String... excludes)
            throws SftpException {
        List<FileAttri> lsFiles = new ArrayList<>();
        FileAttri tmpFileAttri;
        long tmpSize;
        String tmpFullPath;
        Vector<LsEntry> vector = sftp.ls(remotePath);
        for (LsEntry entry : vector) {
            if (".".equals(entry.getFilename()) || "..".equals(entry.getFilename())) {
                continue;
            }
            tmpFullPath = UrlHelper.mergeUrl(remotePath, entry.getFilename());
            if (excludes != null && ArrayUtils.contains(excludes, tmpFullPath)) {
                logger.debug("Ignore directory:{}", tmpFullPath);
                continue;
            }
            tmpFileAttri = new FileAttri();
            tmpFileAttri.setNodeName(entry.getFilename());
            tmpFileAttri.setLastUpdateDate(entry.getAttrs()
                    .getATime());
            tmpFileAttri.setPath(tmpFullPath);

            if (entry.getAttrs()
                    .isDir()) {
                tmpFileAttri.setDir(true);
                if (descendant) {
                    try {
                        List<FileAttri> childs = ls(sftp, tmpFileAttri.getPath(), descendant, excludes);
                        if (CollectionHelper.isNotEmpty(childs)) {
                            tmpFileAttri.addNodes(childs);
                        }
                    } catch (PermissionException e) {
                        tmpFileAttri.setNodeName(tmpFileAttri.getNodeName() + "[No permission]");
                    }
                }
            } else {
                tmpFileAttri.setDir(false);
                tmpSize = entry.getAttrs()
                        .getSize();
                if (tmpSize < 1024) {
                    tmpFileAttri.setSize(entry.getAttrs()
                            .getSize() + "B");
                } else if (tmpSize >= 1024 && tmpSize < 1048576) {
                    tmpFileAttri.setSize(MathHelper.round((entry.getAttrs()
                            .getSize() / 1024f), 1) + "KB");
                } else if (tmpSize > 1048576) {
                    tmpFileAttri.setSize(MathHelper.round((entry.getAttrs()
                            .getSize() / 1048576f), 2) + "MB");
                }
            }
            lsFiles.add(tmpFileAttri);
        }
        return lsFiles;
    }

    /**
     * Determine whether the file exists
     * 
     * @param filePath
     * @return
     * @throws RemoteAccessException
     */
    public boolean isExist(String filePath) throws RemoteAccessException {
        ChannelSftp sftp = null;
        try {
            sftp = openChannel();
            SftpATTRS attrs = sftp.lstat(filePath);
            return attrs != null;
        } catch (SftpException e) {
            logger.error("file does not exist,remotePath:{} , error:{}", filePath, e.getMessage());
            return false;
        }
    }

    @Override
    public void close() throws IOException {
        if (channel != null) {
            try {
                channel.disconnect();
            } catch (Exception e) {
                channel = null;
            }
        }
        super.close();
    }
}
Click and drag to move

Step 3: create ssh interactive tool class: sshclient Java to realize the functions of executing a single command and executing commands asynchronously: the code is as follows: huo get

import com.zjh.logviewer.model.Server;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;

import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSchException;

public class SSHClient extends BaseJchClient {

    public SSHClient(Server serverInfo) throws RemoteAccessException {
        super(serverInfo);
    }

    /**
     * Execute a single command
     * 
     * @param cmd
     * @return StringBuffer command result
     * @throws RemoteAccessException
     */
    public StringBuffer exec(String cmd) throws RemoteAccessException {
        String line = null;
        BufferedReader reader = null;
        ChannelExec channelExec = null;
        StringBuffer resultBuf = new StringBuffer();
        InputStream inStream = null;
        try {
            channelExec = getChannel(cmd);
            stopLastCmdThread();// Interrupts the execution of the previous asynchronous command
            inStream = channelExec.getInputStream();
            channelExec.connect();
            if (inStream != null) {
                reader = IOHelper.toBufferedReader(inStream, StandardCharsets.UTF_8.name());
                while ((line = reader.readLine()) != null) {
                    resultBuf.append(line)
                            .append(Chars.LF);
                }
            }
        } catch (IOException | JSchException e) {
            logger.error("Execute command exception,ip:{},cmd:{},ex:{}", serverInfo.getIp(), cmd, e);
            throw new RemoteAccessException(e);
        } finally {
            IOHelper.closeQuietly(inStream);
            if (channelExec != null) {
                channelExec.disconnect();
            }
        }
        if (logger.isDebugEnabled()) {
            logger.debug("{}Execute command:{},result:{}", serverInfo.getIp(), cmd, resultBuf);
        }
        return resultBuf;
    }

    ExecAsyncCmdThread cmdThread = null;

    /**
     * Execute commands asynchronously
     * 
     * @param cmd
     * @param handler
     * @throws RemoteAccessException
     */
    public void execAsync(final String cmd, final AsyncCmdCallBack callBack) throws RemoteAccessException {
        ChannelExec channelExec = getChannel(cmd);
        stopLastCmdThread();
        cmdThread = new ExecAsyncCmdThread(channelExec, callBack);
        cmdThread.setDaemon(true);
        cmdThread.start();
    }

    private ChannelExec getChannel(String cmd) throws RemoteAccessException {
        conn();
        try {
            ChannelExec channel = (ChannelExec) session.openChannel("exec");
            channel.setCommand(cmd);
            channel.setInputStream(null);
            channel.setErrStream(System.err);
            return channel;
        } catch (JSchException e) {
            throw new RemoteAccessException(e);
        }
    }

    class ExecAsyncCmdThread extends Thread {

        private AsyncCmdCallBack callBack;
        private ChannelExec channelExec = null;
        private volatile boolean isRunning = false;

        public ExecAsyncCmdThread(final ChannelExec channelExec, AsyncCmdCallBack callBack) {
            this.channelExec = channelExec;
            this.callBack = callBack;
        }

        @Override
        public void run() {
            InputStream inStream = null;
            String line;
            BufferedReader reader = null;
            channelExec.setPty(true);
            try {
                inStream = channelExec.getInputStream();
                channelExec.connect();
                isRunning = true;
                reader = IOHelper.toBufferedReader(inStream, StandardCharsets.UTF_8.name());
                while (isRunning) {
                    while ((line = reader.readLine()) != null) {
                        callBack.hanndle(line);
                    }
                    if (channelExec.isClosed()) {
                        int res = channelExec.getExitStatus();
                        isRunning = false;
                        System.out.println(String.format("Exit-status: %d,thread:%s", res, Thread.currentThread()
                                .getId()));
                        break;
                    }
                }
            } catch (Exception e) {
                logger.error("", e);
            } finally {
                IOHelper.closeQuietly(reader);
                if (channelExec != null) {
                    channelExec.disconnect();
                }
                isRunning = false;
            }
        }

        public void close() {
            channelExec.disconnect();
            isRunning = false;
        }

    }

    void stopLastCmdThread() {
        if (cmdThread != null) {
            cmdThread.close();
        }
    }

    @Override
    public void close() throws IOException {
        stopLastCmdThread();
        cmdThread = null;
        super.close();
    }

}

Step 4: call these two classes where you need to use them

Get project source code

https://github.com/BigDataAiZq/logviewer

Programmer Think

java implementation of SFTP and SSH

Introduction to jsch

Get project source code

Hot Topics