TUS is an open source continuation framework for breakpoints on github. The server side is written in go, and the client side supports many kinds. It supports js, andorid, java and IOS.This article will do some analysis on the android side.

Let's start with the core part of the framework, which is how to continue.When it comes to continuation, the upload continues where it was last uploaded, instead of starting from scratch.TUS does this by uploading the necessary parameters when the client creates the link, and the server finds the corresponding file based on the parameters uploaded by the client, and returns the location of the last upload to the client.Clients upload based on the offset returned by the service.

What parameters will the client pass?What parameters are passed by the client to ensure that the file found by the server is the corresponding file?File name and size are used here as file uniqueness.If the file names are the same and the file sizes are the same, then the server considers the same file.

        try {
            SharedPreferences pref = getSharedPreferences("tus", 0);
            client = new TusClient();
            client.setUploadCreationURL(new URL("http://192.168.160.1:1080/files/"));
            client.enableResuming(new TusPreferencesURLStore(pref));
        } catch(Exception e) {
            showError(e);
        }

First, create a client object, pass in the address to upload, and set the preference for the uploaded information.

    public void resumeUpload() {
        try {
            TusUpload upload = new TusAndroidUpload(fileUri, this);
            uploadTask = new UploadTask(this, client, upload);
            uploadTask.execute(new Void[0]);
        } catch (Exception e) {
            showError(e);
        }
    }

This code creates the uploaded object and puts the uploaded task into a thread to execute.This is where the information for the file comes in.

 protected URL doInBackground(Void... params) {
            try {
                TusUploader uploader = client.resumeOrCreateUpload(upload);
                long totalBytes = upload.getSize();
                long uploadedBytes = uploader.getOffset();

                // Upload file in 1MiB chunks
                uploader.setChunkSize(1024 * 1024);
                Log.i("upload","upload_tus start");
                while(!isCancelled() && uploader.uploadChunk() > 0) {
                    uploadedBytes = uploader.getOffset();
                    Log.i("upload","upload_tus byte = "+uploadedBytes);
                    publishProgress(uploadedBytes, totalBytes);
                }
                Log.i("upload","upload_tus finish");
                uploader.finish();
                return uploader.getUploadURL();

            } catch(Exception e) {
                exception = e;
                cancel(true);
            }
            return null;
        }

This is the upload process, first get an uploader object, in which the interaction with the server has been completed, and get the offset.uploader.setChunkSize This method sets the size of the file to be read once.The larger the setting, the more bytes read at a time.Uploader.uploadChunk() is the real upload place.A while loop is used here to read and upload files continuously.The purpose of publishProgress is to show the user a progress bar.Let's look at the implementation of uploader.uploadChunk().

public int uploadChunk() throws IOException, ProtocolException {
        openConnection();

        int bytesToRead = Math.min(getChunkSize(), bytesRemainingForRequest);

        int bytesRead = input.read(buffer, bytesToRead);
        if(bytesRead == -1) {
            // No bytes were read since the input stream is empty
            return -1;
        }

        // Do not write the entire buffer to the stream since the array will
        // be filled up with 0x00s if the number of read bytes is lower then
        // the chunk's size.
        output.write(buffer, 0, bytesRead);
        output.flush();

        offset += bytesRead;
        bytesRemainingForRequest -= bytesRead;

        if(bytesRemainingForRequest <= 0) {
            finishConnection();
        }

        return bytesRead;
    }

First open the link here, and then return if the link is already open.Then read the data from the file, write it to the output stream, and refresh the buffer.Then add up the offsets.Later sections of code are not very understood.BytesRemainingForRequest This is the size of a request sent. If a link sends more requests than this number, the request will be disconnected and re-linked when it is looped again.You will be bored with this. Why do you want to re-link it?This is actually related to the inputstream type used by the client.If bufferedinputstream is used, this type of inputstream caches the file bytes read in order to reduce the number of operations on the file. The more bytes read, the larger the cache will be, which will necessarily result in large memory requests.An oom may occur if the requested memory is large enough.So to solve this problem, bytesRemainingForRequest is used to control the size of the bufferedinputstream cache.Although this reduces access to files, it is not preferable after my actual tests. Each time a link is broken and then re-linked, it may cause tcp reset exceptions or IO errors during transmission.Especially when uploading large files, there are several disconnections and links.For this reason, I discard the use of bufferedinputstream here.Modify the following code:

 public TusInputStream(InputStream stream) {
       /* if(!stream.markSupported()) {
            stream = new BufferedInputStream(stream);
        }*/

        this.stream = stream;
    }

The code masked above allows it to not use bufferedinputstream.And remove the condition for disconnecting the link:

 public int uploadChunk() throws IOException, ProtocolException {
        openConnection();

        int bytesToRead = Math.min(getChunkSize(), bytesRemainingForRequest);

        int bytesRead = input.read(buffer, bytesToRead);
        if(bytesRead == -1) {
            // No bytes were read since the input stream is empty
            return -1;
        }

        // Do not write the entire buffer to the stream since the array will
        // be filled up with 0x00s if the number of read bytes is lower then
        // the chunk's size.
        output.write(buffer, 0, bytesRead);
        output.flush();

        offset += bytesRead;
        /*
        bytesRemainingForRequest -= bytesRead;

        if(bytesRemainingForRequest <= 0) {
            finishConnection();
        }
        */
        return bytesRead;
    }

bytesRemainingForRequest will no longer be used as a condition for closing links.