Egret (wechat) resource CRC management

Posted by Floodboy on Fri, 28 Jan 2022 23:15:30 +0100

1, CRC - Cyclic Redundancy Check

Because of my lack of talent and learning, I won't explain the principle here. Please Baidu for the detailed principle~

2, Generate CRC suffix in Egret

egret is in config The corresponding interface is provided in the TS file. You can see the judgment of else if (command = = 'publish') in the following code, which is the open extension of the publish command. We add the CRC suffix to the project resources through renameplugin.

These are detailed descriptions of the parameters in the construction
Among them, matchers is the matching mechanism, which can be named by ourselves

  • Files that meet the requirements of from are output as files in to format
  • from uses glob expression, and to contains four variables [path][name][hash][ext] (we can add separators we need between the four variables, such as -)
    It should be noted that hash egret currently only supports crc32
type RenamePluginOptions = {

        /**
         * Output log
         * Whether to output the log
         */
        verbose?: boolean

        /**
         * What kind of hash algorithm is used? At present, only crc32 is supported
         * What hash algorithm is used, currently only crc32 is supported
         */
        hash?: "crc32"


        /**
         * Set matching rules to rename the specified file
         * This parameter is an array that allows multiple matching rules to be set
         * Set up matching rules to copy specified files to other folders
         * This parameter is an array that allows multiple matching rules to be set
         */
        matchers: Matcher[]

        /**
         * Callback function. The return value includes some information of the file
         * The callback function, return value includes some information about the file
         */
        callback?: Function
    }

After the above modifications, you can enter egret publish on the Egret console to try packaging. If verbose is set to true, you can see the renamed output on the console

3, Generate version file

After the above operation, the typed resources will have the suffix we want, but we also need a version file to compare the resources

1. Add a custom plug-in in config Create your own plug-in class under the TS peer, in config ts

You can add it to the database in the above way**commands**Implemented in

ResPlugin should implement plugins Command interface. Here is the declaration of the interface

    /**
     * Build pipeline command
     */
    interface Command {

        /**
         * This function will be executed for each file in the project. Returning file means to keep the file, and returning null means to delete the file from the build pipeline, that is, it will not be published
         */
        onFile?(file: File): Promise<File | null>

        /**
         * After all files in the project are executed, this function will eventually be executed.
         * This function is mainly used to create new files
         */
        onFinish?(pluginContext?: CommandContext): Promise<void>

        [options: string]: any;
    }

2. Implement ResPlugin plug-in

The onFile method can be used to filter files, and unnecessary files can return null
Take the plug-in named ResPlugin as an example

export class ResPlugin implements plugins.Command { 
 
    // file information 
    private versionConfig = {}; 
    // Used to determine whether the resource is in this directory 
    private asstesVersionPath = "resource/asset/"; 
    // Output file path 
    private versionConfigPath = "resource/version"; 
    // Output file extension 
    private versionConfigExtname = ".json"; 
    // Intercepted text version number 
    private versionNo: string; 
 
    constructor() { 
    } 
 
    async onFile(file: plugins.File) { 
        // File extension 
        const extname = file.extname; 
        // Initial file name 
        const path = file.origin; 
        const _index = file.path.lastIndexOf("\\"); 
        // Real file name 
        const _path = file.path.substring(_index + 1); 

        if (path.indexOf(this.asstesVersionPath) != -1 && (extname === ".mp3" || extname === ".fnt" || extname === ".json" || extname === ".png" 
            || extname === ".jpg" || extname === ".xml" || extname === ".dbbin" || extname === ".txt" || extname === ".dat" || extname === ".bin") 
            || extname === ".json") { 
            this.versionConfig[path] = _path; 
        } else { 
            // Intercept game version number 
            if (path.indexOf('jslauncher.js') != -1 && !this.versionNo) { 
                let jslauncher = file.contents.toString();
                let arr = jslauncher.match('(?<=versionNo.*=.*").*?(?=\")');
                this.versionNo = arr ? arr[0] : '';
            } 
        } 
        return file; 
    } 
 
    async onFinish(commandContext: plugins.CommandContext) { 
        let fileName = this.versionConfigPath + this.versionNo + this.versionConfigExtname; 
        commandContext.createFile(fileName, new Buffer(JSON.stringify(this.versionConfig))); 
    } 
} 

3. Call ResPlugin plug-in

Go back to config TS, import {resplugin} from '/ ResPlugin’; Import resplugin and add it in commands
It should be noted that the from is followed by a relative path, and there is no need to write a file suffix

4, CRC addition completed

After the above busy work, you can package it to see the effect. You can package it with command and Egret Launcher

5, Use of version file

1. Custom version controller

To implement RES.IVersionController interface, we need to delete old files through these two interfaces
The interface has two methods

/**
 * Get version information data< br/>
 * This method needs to be called before the application loads any resources. It is recommended to call it first at the beginning of the application's entry class (Main). This method is only responsible for obtaining version information, not downloading resources.
 * @version Egret 2.4
 * @platform Web,Native
 * @language zh_CN
 */
init(): Promise<any>;

/**
 * Get the actual URL address of the resource file< br/>
 * Since version control has actually changed the URL of the original resource file, this method needs to be called when you want to obtain the actual URL of the specified resource file< br/>
 * In the development and debugging phase, this method will directly return the passed in parameter value.
 * @param url url used in the game
 * @returns Actually loaded url
 * @version Egret 2.4
 * @platform Web,Native
 * @language zh_CN
 */
getVirtualUrl(url: string): string;

Idea: get the latest version on the network in init JSON file and read the local version JSON file, compare the two files to get the old files that need to be deleted, then delete them and put the latest version The JSON file is written locally.

a. Get the latest version Josn file

// This function is called when the game starts loading resources
init(): Promise<any> {
    if (RELEASE) {//Release mode
        return new Promise((resolve, reject) => {
            // Here, the configuration resource information is obtained through httpReques
            let request = new egret.HttpRequest();
            request.responseType = egret.HttpResponseType.TEXT;
            request.open(this.versionConfigPath + "?v=" + Math.random(), egret.HttpMethod.GET);
            request.send();
            request.addEventListener(egret.Event.COMPLETE, (event: egret.Event) => {
                var request = <egret.HttpRequest>event.currentTarget;
                try {
                    this.versionConfig = JSON.parse(request.response);
                    // Handle caching of file resources
                    // The following is mainly about the version control of various runtime,
                    if (egret.Capabilities.runtimeType == egret.RuntimeType.WEB) {
                        this.versionCacheWeb();//Browser H5
                    } else {
                        this.versionCacheWxgame();
                    }
                }
                catch (e) {
                    Main.onRemoveSubPackLoading();
                    console.error("versionConfig error:" + e);
                }
                resolve();
            }, this);
            request.addEventListener(egret.IOErrorEvent.IO_ERROR, () => {
                //egret.log("post error : " + event);
                Main.onRemoveSubPackLoading();
                resolve();
            }, this);
        })
    } else {
        return Promise.resolve()
    }
}

b. Get the resource path to delete:

Obtain the last resource version according to the local cache, judge whether there is any version change, delete all if any, and then write the new version JSON file, get the resource path to be deleted without change

/**
 * Handle invalid resources in wxgame
 */
private versionCacheWxgame = () => {
    // Simple processing: obtain the current resource version number resVersion through localStorage. If the version numbers are different, delete the expired resources
    // Here, developers can control the version number by themselves, not necessarily using localStorage
    try {
        const localVersion = egret.localStorage.getItem("resVersion");
        console.log("resVersion -->> " + localVersion)
        // Resource version change
        if ((localVersion != this.currResourceVersion)) {
            egret.localStorage.setItem("resVersion", this.currResourceVersion);
            ClearLastLocalCache();
            mkdir(this.currResourceVersion);
            writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));

        } else {
            // Check the old resources to delete here
            let oldRes = this.getRemoveList();
            if (oldRes && oldRes.length > 0) {
                // Delete old resource
                for (let i = 0; i < oldRes.length; i++) {
                    deleteLocalFile(oldRes[i]);
                }
            }
        }
    } catch (e) {

    }
    Main.onRemoveSubPackLoading();
}



/** 
* Get the expired resource list of the previous version, improve efficiency, and compare it in the form of string
*/
private getRemoveList(): Array<string> {
    try {
        let rootPath = GetUserDataPath();
        console.log('---------rootPath------ ' + rootPath)
        let versionPath = this.currResourceVersion + "/version" + this.currentVersion + ".json";
        let result = [];
        if (fileExist(versionPath)) {
            // Read local version file
            let content = readFileSync(versionPath, 'utf-8');
            if (!content) {
                // Replace the original version file here
                deleteLocalFile(versionPath);
                writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));
                return result;
            }
            let oldVersion = JSON.parse(content);
            let temp = '';
            for (const key in oldVersion) {
                if (!this.versionConfig[key] || (this.versionConfig[key] != oldVersion[key])) {
                    console.log('key ---- > ' + key) // resource/asset/sound/challenge/match.mp3
                    console.log('value -- > ' + oldVersion[key]) // value -- > match_36ef1cca.mp3
                    // Splice the real name of the file,
                    temp = this.currResourceVersion + key.substring(key.indexOf('/'), key.lastIndexOf('/')) + '/' + oldVersion[key];
                    console.log('Spliced path -->> ' + temp) // 9.9.9.9/asset/sound/challenge/match_36ef1cca.mp3
                    result.push(temp);
                }
            }
            // Replace the original version file here
            deleteLocalFile(versionPath);
            writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));
    
        } else {
            // No old version file is generated directly
            mkdir(this.currResourceVersion);
            writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));
        }
    
        return result;
    
    } catch (e) {
        console.error('getRemoveList -- error')
        return null;
    }
}

c. Implement the getVirtualUrl interface, which is used to obtain the actual URL address of the resource file

//When the game is running, each resource loading url must go through this function
getVirtualUrl(url: string) {
    return this.getResUrlByVersion(url);
}
/**
 * Get the path information after version control
 */
getResUrlByVersion(url: string): string {
    //Judge whether it is version controlled resources, resources in other domains, such as player avatars, or resources in the initial package are loaded with the original url
    if (url.indexOf(this.resourceRoot) == -1) {
        return url;
    }
    console.log('version ---> ' + url) //  https://uqxsstest.pook.com/update/9.9.9.9/resource/asset/view/img/common/common_ui/common_ui.json
    const _url = url;
    //Erase the resourceRoot path of the file, and then judge whether the file has been version managed
    url = url.replace(this.resourceRoot, "");
    if (this.versionConfig) {
        // Some files may exist? In the form of adding numbers to control, adding v = is not supported at the bottom of the engine, and the path will only be loaded with the original url,
        // If a similar situation exists in the project, restore it to normal form
        const index = url.indexOf("?v=");
        if (index != -1) {
            url = url.slice(0, index);
        }

        //Get the version of version control
        let versionKey = this.versionPath + url;
        let _key = url.substring(url.lastIndexOf("/") + 1);
        //console.log("versionKey:" + versionKey);
        if (this.versionConfig[versionKey]) {
            const version = this.versionConfig[versionKey];
            const ext = url.slice(url.lastIndexOf("."));
            // Original folder + crc32 code + suffix extension
            url = url.replace(_key, version);
        } else {
            return _url;
        }
    }
    url = this.resourceRoot + url;
    console.log('version end --> ' + url) // https://uqxsstest.pook.com/update/9.9.9.9/resource/asset/view/img/common/common_ui/common_ui_948ba063.json
    return url;
}

Full code:

declare var wx: { getFileSystemManager, env }
class VersionController implements RES.IVersionController {
    // Version control information
    private versionConfig;
    // resource root path
    private resourceRoot = getImgUrl() + "resource/";
    // Versioned folder
    private versionPath = "resource/";
    // The path where the version control information is located, relative to the resource folder
    private versionConfigPath = getImgUrl() + "/resource/version" + getVersionNo() + ".json";
    // Current version number of wxgame
    private currentVersion = getVersionNo();
    // Current resource version number
    private currResourceVersion = getResourceVersion();

    public constructor() {
    }

    // This function is called when the game starts loading resources
    init(): Promise<any> {
        if (RELEASE) {//Release mode
            return new Promise((resolve, reject) => {
                //Obtain configuration resource information through httpReques
                let request = new egret.HttpRequest();
                request.responseType = egret.HttpResponseType.TEXT;
                request.open(this.versionConfigPath + "?v=" + Math.random(), egret.HttpMethod.GET);
                request.send();
                request.addEventListener(egret.Event.COMPLETE, (event: egret.Event) => {
                    var request = <egret.HttpRequest>event.currentTarget;
                    try {
                        this.versionConfig = JSON.parse(request.response);
                        //Handle caching of file resources
                        // The following is mainly about the version control of various runtime,
                        if (egret.Capabilities.runtimeType == egret.RuntimeType.WEB) {
                            this.versionCacheWeb();//Browser H5
                        } else {
                            this.versionCacheWxgame();
                        }
                    }
                    catch (e) {
                        Main.onRemoveSubPackLoading();
                        console.error("versionConfig error:" + e);
                    }
                    resolve();
                }, this);
                request.addEventListener(egret.IOErrorEvent.IO_ERROR, () => {
                    Main.onRemoveSubPackLoading();
                    resolve();
                }, this);
            })
        } else {
            return Promise.resolve()
        }
    }

    /**
     * Handle invalid resources in wxgame
     */
    private versionCacheWxgame = () => {
        // Simple processing: obtain the current resource version number resVersion through localStorage. If the version numbers are different, delete the expired resources
        // Here, developers can control the version number by themselves, not necessarily using localStorage
        try {
            const localVersion = egret.localStorage.getItem("resVersion");
            // Resource version change
            if ((localVersion != this.currResourceVersion)) {
                egret.localStorage.setItem("resVersion", this.currResourceVersion);
                ClearLastLocalCache();
                mkdir(this.currResourceVersion);
                writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));

            } else {
                // Check the old resources to delete here
                let oldRes = this.getRemoveList();
                if (oldRes && oldRes.length > 0) {
                    // Delete old resource
                    for (let i = 0; i < oldRes.length; i++) {
                        deleteLocalFile(oldRes[i]);
                    }
                }
            }
        } catch (e) {

        }
        Main.onRemoveSubPackLoading();
    }
    /**
    * Handle the updates in WebH5. If the developer has version control released by other platforms, please refer to these two methods for modification
    */
    private versionCacheWeb = () => {
        // Simple processing: obtain the current resource version number resVersion through localStorage. If the version numbers are different, delete the expired resources
        // Here, developers can control the version number by themselves, not necessarily using localStorage
        const localVersion = egret.localStorage.getItem("resVersion");
        if (localVersion != this.currResourceVersion) {
            egret.localStorage.setItem("resVersion", this.currResourceVersion);
            console.log("Web Version update");
        }
    }
    /** 
     * Get the expired resource list of the previous version, improve efficiency, and compare it in the form of string
     */
    private getRemoveList(): Array<string> {
        try {
            let rootPath = GetUserDataPath();
            let versionPath = this.currResourceVersion + "/version" + this.currentVersion + ".json";
            let result = [];
            if (fileExist(versionPath)) {
                let content = readFileSync(versionPath, 'utf-8');
                if (!content) {
                    // Replace the original version file here
                    deleteLocalFile(versionPath);
                    writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));
                    return result;
                }
                let oldVersion = JSON.parse(content);
                let temp = '';
                for (const key in oldVersion) {
                    if (!this.versionConfig[key] || (this.versionConfig[key] != oldVersion[key])) {
                        temp = this.currResourceVersion + key.substring(key.indexOf('/'), key.lastIndexOf('/')) + '/' + oldVersion[key];
                        result.push(temp);
                    }
                }
                // Replace the original version file here
                deleteLocalFile(versionPath);
                writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));

            } else {
                // No old version file is generated directly
                mkdir(this.currResourceVersion);
                writeFile(this.currResourceVersion + "/version" + this.currentVersion + ".json", JSON.stringify(this.versionConfig));
            }
            return result;
        } catch (e) {
            console.error('getRemoveList -- error')
            return null;
        }
    }
    //When the game is running, each resource loading url must go through this function
    getVirtualUrl(url: string) {
        return this.getResUrlByVersion(url);
    }
    /**
     * Get the path information after version control
     */
    getResUrlByVersion(url: string): string {
        //Judge whether it is version controlled resources, resources in other domains, such as player avatars, or resources in the initial package are loaded with the original url
        if (url.indexOf(this.resourceRoot) == -1) {
            return url;
        }
        const _url = url;
        //Erase the resourceRoot path of the file, and then judge whether the file has been version managed
        url = url.replace(this.resourceRoot, "");
        if (this.versionConfig) {
            // Some files may exist? In the form of adding numbers to control, adding v = is not supported at the bottom of the engine, and the path will only be loaded with the original url,
            // If a similar situation exists in the project, restore it to normal form
            const index = url.indexOf("?v=");
            if (index != -1) {
                url = url.slice(0, index);
            }
            //Get the version of version control
            let versionKey = this.versionPath + url;
            let _key = url.substring(url.lastIndexOf("/") + 1);
            if (this.versionConfig[versionKey]) {
                const version = this.versionConfig[versionKey];
                const ext = url.slice(url.lastIndexOf("."));
                // Original folder + crc32 code + suffix extension
                url = url.replace(_key, version);
            } else {
                return _url;
            }
        }
        url = this.resourceRoot + url;
        return url;
    }
}

2. Register version controller

According to their suggestion, we are in main Register the self implemented manager * * RES.registerVersionController(new VersionController()) in the constructor of TS** That's it

Topics: TypeScript