Great Webpack Construction Process Learning Guide

Posted by sambkk on Mon, 22 Jun 2020 02:25:01 +0200

Recent original articles review:

Webpack is a hot front-end packaging tool and is essentially a static module bundler for modern JavaScript applications.When a Webpack processes an application, it recursively builds a dependency graph that contains each module the application needs, then packages all the modules into one or more bundles.

In fact, Webpack is a JS code packer.

As for pictures, CSS, Less, TS and other files, you need Webpack with loader or plugin function to achieve ~

1. Webpack Construction Process Analysis

1. Webpack building process

First, let's take a brief look at the Webpack building process:

Identify the entry file according to the configuration;
Layer-by-layer module dependencies (including Commonjs, AMD, or ES6 import s) are identified and analyzed;
The main work of Webpack is to analyze code, convert code, compile code, and output code.
Output the last packaged code.

2. Webpack construction principles

After reviewing the above brief introduction to the construction process, I believe you have a simple understanding of this process, so let's start with a detailed description of the Webpack construction principles, including a series of processes from start-up to output:

(1) Initialization parameters

Parse Webpack configuration parameters, merge Shell incoming andWebpack.config.jsFile configuration parameters to form the final configuration result.

(2) Start compilation

The parameters from the previous step initialize the compiler object, register all configured plug-ins, listen to event nodes for the Webpack build lifecycle, react accordingly, and execute the object's run method to start compiling.

(3) Identify entrance

From the configuration file (webpack.config.js) the entry entry specified in the file, start parsing the file to build the AST grammar tree, find out the dependencies, and go on recursively.

(4) Compile modules

Recursively, according to the file type and loader configuration, calls all the configured loaders to convert the file, finds the module that the module depends on, and recursively this step until all the entrance dependent files have been processed by this step.

(5) Complete module compilation and output

After recursion, each file result is obtained, including each module and their dependencies, and the code block chunk is generated according to the entry configuration.

(6) Output complete

Output all chunk s to the file system.

Note: There are a series of plug-ins in the build lifecycle that do the right thing at the right time, such as UglifyPlugin, which results before the loader transform recursively overrides the results with UglifyJs compression.

2. Handwritten Webpack Construction Tools

At this point, I'm sure you already know about the Webpack building process, but that's not enough. Let's start writing the Webpack building tools to apply what's described above to the actual code, so let's get started.

1. Initialize the project

Before we start with a handwriting build tool, we'll initialize a project:

$ yarn init -y

And install the following four dependent packages:

@babel/parser:Used for analysis byFs.readFileSyncRead the contents of the file and return AST (Abstract Grammar Tree);
@babel/traverse: Used to traverse the AST to obtain the necessary data;
@babel/core: The Babel core module, which provides the transformFromAst method for converting AST into browser-enabled code;
@babel/preset-env: Convert the converted code to ES5 code;

$ yarn add @babel/parser @babel/traverse @babel/core @babel/preset-env

Initialize project directories and files:

Code stored in warehouse: https://github.com/pingan8787/Leo-JavaScript/tree/master/Cute-Webpack/Write-Webpack

Since the core content of this section is to implement the Webpack building tools, the following steps start with the'(3) Determine Entry'step in 2. Webpack Building Principles.

The general code implementation process is as follows:

As you can see from the diagram, the core of handwritten Web packs is to implement the following three methods:

createAssets: Code for collecting and processing files;
createGraph: Returns all file dependency graphs based on the entry file;
bundle: The entire code is output according to the dependency graph;

2. Implement the createAssets function

2.1 Read through the entry file and convert to AST

Start by writing some simple code in the. /src/index file:

// src/index.js

import info from "./info.js";
console.log(info);

Implement file read and AST conversion operations in the createAssets method:

// leo_webpack.js

const fs = require("fs");
const path = require("path");
const parser = require("@babel/parser");
const traverse = require("@babel/traverse").default;
// Due to the ES Module export used by traverse, we add A. default if introduced by requier
const babel = require("@babel/core");

let moduleId = 0;
const createAssets = filename => {
    const content = fs.readFileSync(filename, "utf-8"); // Read the file stream synchronously based on the file name
  
      // Convert read file stream buffer to AST
    const ast = parser.parse(content, {
        sourceType: "module" // Specify Source Type
    })
    console.log(ast);
}

createAssets('./src/index.js');

Code above:
adoptFs.readFileSync() method, reads the file stream under the specified path synchronously, and converts the read file stream buffer into browser-aware code (AST) by parser relying on the parse() method provided by the package, with the AST output as follows:

Also note that here we declare a moduleId variable to distinguish the modules that are currently operating on.
Here, not only will the read file stream buffer be converted to AST, but also the ES6 code to ES5 code.

2.2 Collect dependencies for each module

Next, the dependencies variable is declared to hold the collected file dependency paths, traverse ast through the traverse () method to get each node dependency path, and push into the dependencies array.

// leo_webpack.js

function createAssets(filename){
    // ...
    const dependencies = []; // Path used to collect File Dependencies

      // Get the dependent paths for each node by manipulating the AST provided by traverse
    traverse(ast, {
        ImportDeclaration: ({node}) => {
            dependencies.push(node.source.value);
        }
    });
}

2.3 Convert AST to Browser Runnable Code

While collecting dependencies, we can convert AST code into browser runnable code, which requires the use of babel, a versatile little fellow who provides us with a very useful transformFromAstSync() method to synchronously convert AST to browser runnable code:

// leo_webpack.js

function createAssets(filename){
    // ...
    const { code } = babel.transformFromAstSync(ast,null, {
        presets: ["@babel/preset-env"]
    });
    let id = moduleId++; // Set the current processing module ID
    return {
        id,
        filename,
        code,
        dependencies
    }
}

At this point, we are executing node leo_webpack.js, outputs the following, including the path filename for the entry file, the browser executable code, and the path dependencies array on which the file depends:

$ node leo_webpack.js

{ 
  filename: './src/index.js',
  code: '"use strict";\n\nvar _info = _interopRequireDefault(require("./info.js"));\n\nfunction _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { "default": obj }; }\n\nconsole.log(_info["default"]);', 
  dependencies: [ './info.js' ] 
}

2.4 Code Summary

// leo_webpack.js

const fs = require("fs");
const path = require("path");
const parser = require("@babel/parser");
const traverse = require("@babel/traverse").default;
// Due to the ES Module export used by traverse, we add A. default if introduced by requier
const babel = require("@babel/core");

let moduleId = 0;
function createAssets(filename){
    const content = fs.readFileSync(filename, "utf-8"); // Read the file stream synchronously based on the file name
  
      // Convert read file stream buffer to AST
    const ast = parser.parse(content, {
        sourceType: "module" // Specify Source Type
    })
    const dependencies = []; // Path used to collect File Dependencies

      // Get the dependent paths for each node by manipulating the AST provided by traverse
    traverse(ast, {
        ImportDeclaration: ({node}) => {
            dependencies.push(node.source.value);
        }
    });

      // Convert ES6 code to ES5 code via AST
    const { code } = babel.transformFromAstSync(ast,null, {
        presets: ["@babel/preset-env"]
    });
  
    let id = moduleId++; // Set the current processing module ID
    return {
          id,
        filename,
        code,
        dependencies
    }
}

3. Implement the createGraph function

In the createGraph() function, we will recursively analyze all dependent modules, cycle through each dependent module dependency, and generate a dependency map.
For testing convenience, we addConsts.jsandInfo.jsFile code, add some dependencies:

// src/consts.js
export const company = "Safety";

// src/info.js
import { company } from "./consts.js";
export default `Hello, ${company}`;

Next, you begin to implement the createGraph() function, which takes the path to an entry file as a parameter:

// leo_webpack.js

function createGraph(entry) {
    const mainAsset = createAssets(entry); // Get the contents under the entry file
    const queue = [mainAsset]; // The result of the entry file as the first item
    for(const asset of queue){
        const dirname = path.dirname(asset.filename);
        asset.mapping = {};
        asset.dependencies.forEach(relativePath => {
            const absolutePath = path.join(dirname, relativePath); // Convert file path to absolute path
            const child = createAssets(absolutePath);
            asset.mapping[relativePath] = child.id; // Save Module ID 
            queue.push(child); // Files that recursively traverse all child nodes
        })
    }
    return queue;
}

Code above:

The contents of the entry file are first read through the createAssets() function and used as the first item of the queue array of dependencies (dependency map), then each item of the dependency map queue is iterated through, then the dependency dependencies array in each item is iterated over, and each item of the dependency is stitched together into an absolute path of the dependency (absolutePath) as createAssets().The parameters of the function call recursively iterate through the files of all the child nodes and save the results in the dependency map queue.

Note that the mapping object is the relationship between the relative path of the file and the module ID. In the mapping object, we use the relative path of the dependent file as the key to store the module ID.

Then we modify the startup function:

// leo_webpack.js

- const result = createAssets('./src/index.js');
+ const graph = createGraph("./src/index.js");
+ console.log(graph);

Then we get a dependency map that contains all the file dependencies:

This dependency map contains the dependencies of all file modules as well as the code content of the modules.The next step is to simply implement the bundle() function and output the result.

4. Implement bundle functions

From the previous discussion, we know that the function createGraph() returns a dependency map queue containing each dependency related information (id / filename / code / dependencies), which will be used in this step.

In the bundle() function, a dependency graph is received as a parameter, and the compiled result is output.

4.1 Read all module information

We first declare a variable module with a value of string type, then iterate through the parameter graph, using the id attribute in each item as a key and an array of values, including a method to execute code and a serialized mapping, and finally stitch it into modules.

// leo_webpack.js

function bundle(graph) {
    let modules = "";
    graph.forEach(item => {
        modules += `
            ${item.id}: [
                function (require, module, exports){
                    ${item.code}
                },
                ${JSON.stringify(item.mapping)}
            ],
        `
    })
}

Code above:

In the value of each item in modules, the element with a subscript of 0 is a function that receives three parameters, require / module / exports. Why do you need these three parameters?

The reason is that the build tool cannot determine if the require / module / exports three module methods are supported, so you need to implement them yourself (later steps will do so) before the code within the method can execute properly.

4.2 Return to Final Results

Next, we implement the bundle() function return value processing:

// leo_webpack.js

function bundle(graph) {
    //...
    return `
        (function(modules){
            function require(id){
                const [fn, mapping] = modules[id];
                function localRequire(relativePath){
                    return require(mapping[relativePath]);
                }

                const module = {
                    exports: {}
                }

                fn(localRequire, module, module.exports);

                return module.exports;
            }
            require(0);
        })({${modules}})
    `
}

Code above:

The final bundle function returns a string containing one Self-Executing Function (IIFE) Where the function parameter is an object, key is modules, value is the preceding stitched modules string, that is, {modules: modules string}.

In this self-executing function, the require method is implemented, receiving an ID as a parameter, and within the method, the localRequire / module /Modules.exportsThree methods are passed as parameters to the fn method in the modules[id], and the require() function (require(0);) is finally initialized.

4.3 Code Summary

// leo_webpack.js

function bundle(graph) {
    let modules = "";
    graph.forEach(item => {
        modules += `
            ${item.id}: [
                function (require, module, exports){
                    ${item.code}
                },
                ${JSON.stringify(item.mapping)}
            ],
        `
    })
    return `
        (function(modules){
            function require(id){
                const [fn, mapping] = modules[id];
                function localRequire(relativePath){
                    return require(mapping[relativePath]);
                }

                const module = {
                    exports: {}
                }

                fn(localRequire, module, module.exports);

                return module.exports;
            }
            require(0);
        })({${modules}})
    `
}

5. Execute Code

When all of the above approaches have been implemented, let's try it out:

// leo_webpack.js

const graph = createGraph("./src/index.js");
const result = bundle(graph);
console.log(result)

Now you can see that the terminal outputs code like this, a string, which is copied to the console for easy viewing:

This is the packaged code.

So how do you get this code to execute?Using eval() method:

// leo_webpack.js

const graph = createGraph("./src/index.js");
const result = bundle(graph);
eval(result);

At this time you can see the console output Hello, safe.So let's finish a simple Webpack building tool ~

I'd like to compliment your friends who can see here.

3. Summary

This paper mainly introduces the construction process and principle of Webpack. On this basis, we share the implementation process of handwritten Webpack with you. I hope you can have a better understanding of the Webpack construction process, after all, interviewers like to ask ~

Author	Wang Ping'an
E-mail	pingan8787@qq.com
Blog	www.pingan8787.com
WeChat	pingan8787
Daily article recommendation	https://github.com/pingan8787...
ES Brochure	js.pingan8787.com
Sparrow Knowledge Base	Cute-FrontEnd

Topics: Javascript Webpack github JSON

Programmer Think

Great Webpack Construction Process Learning Guide

1. Webpack Construction Process Analysis

1. Webpack building process

2. Webpack construction principles

2. Handwritten Webpack Construction Tools

1. Initialize the project

2. Implement the createAssets function

2.1 Read through the entry file and convert to AST

2.2 Collect dependencies for each module

2.3 Convert AST to Browser Runnable Code

2.4 Code Summary

3. Implement the createGraph function

4. Implement bundle functions

4.1 Read all module information

4.2 Return to Final Results

4.3 Code Summary

5. Execute Code

3. Summary

Hot Topics