Why do I need a loader

webpack is a static module packaging tool for modern JavaScript applications. Internally, one or more bundle static resources are generated by building the dependency relationship between dependency graph management modules.


However, webpack can only handle JavaScript and Json modules. In addition to JavaScript and Json modules, the application also has media resources such as pictures, audio and fonts, and non js code modules such as style files such as less and sass. Therefore, we need an ability to parse non js resource modules into modules that can be managed by webpack. This is the function of loader.


For example, for the less style file, if index. Is processed in the webpack configuration file The less file will be processed by less loader, CSS loader and style loader, as shown in the following code:

module.exports = {
  module: {
    rules: [
      {
        test: /\.less$/,
        use: ['style-loader', 'css-loader', 'less-loader']
      }
    ]
  }
};

Webpack parses to index When using the less module, you will first use a module similar to FS Readfile to read the file and obtain the source code text source in the file; The obtained source needs to be converted to ast through js parser, but before that, I will go to the loader configured by webpack to see if there is a loader to process the file. It is found that there are ['style loader', 'CSS loader', 'less loader] three loaders to process in order, Therefore, webpack will hand over the source code and loader processor to the loader runner, which will process the source files through layers of loaders according to certain rules, and finally get the modules that can be recognized by webpack; Then it is converted to ast for further processing, such as analyzing the AST and collecting the dependency of the module until the dependent link is analyzed.


So far, you should know index The less source file will be processed by three loaders according to certain rules to get the js module. What do the three loaders do to make it possible to convert from a style file to a js file?


First, the source will be processed as an input parameter by the less loader, which can convert the less code into css code through the less parsing generator. Of course, the converted css code can't be used directly, because import depends on other css files in css.


Pass the css code parsed by less loader into css loader. css parser parsing, that is, postcss parsing, will be used in css loader. For example, import will be parsed into the form of require in js to reference other style resources. At the same time, the css code will be converted into a string through module Exports is thrown. At this time, the css file has been converted into a js module, and the webpack can handle it. However, it cannot be used because it is not referenced as a style tag. Therefore, it needs to be processed by style loader.


Pass the js code parsed by css loader into style loader, process the required path through the path conversion function in loader utils, add and create the style tag, and assign the code referenced by require to innerHtml. In this way, a js code is obtained, which contains the content of the created style tag added by style loader, The content of the tag is processed by css loader, which parses the css into js code, and the less loader parses the less file into css. Then, the less module is parsed into a js module, and the webpack will be managed in a unified way.


This is the process of webpack processing less files into js files, but this is only a small part. If it can be used, it still needs a lot of way to go, but it is not the focus of this article. By now, you should have a general understanding of the role of loader in webpack and why laoder is needed. Simply put, the loader handles modules (modules and files), which can be processed into a way that webpack can parse. At the same time, it can also reprocess the parsed files.


Next, it mainly introduces how to configure loader in webpack; Talk about the working principle of loader from the macro level; At the same time, take it with you to implement the key module loader runner in the loader. Finally, I will lead you to manually write the style loader, CSS loader and less loader mentioned above.

How to configure loader

The following is the basic configuration of loader in webpack:

module.exports = {
  resolveLoader: {
    // Find the loader from the file under the root directory
    modules: ['node_modules', path.join(__dirname, 'loaders')],
  },
  module: {
    rules: [{
        enforce: 'normal',
        test: /\.js$/,
        use: [{
          loader: 'babel-loader',
          options: {
            presets: [
              "@babel/preset-env"
            ]
          }
        }]
      },
      {
        enforce: 'pre',
        test: /\.css$/,
        use: ['style-loader', 'css-loader']
      }
    ]
  }
};

For details, please refer to https://webpack.docschina.org/configuration/module/#rule The rule documentation is described in detail. The more important field is enforce. Divide loader s into post, normal and pre types.


In addition to setting the loader in the configuration file, the loader is the processing of any file or module. Therefore, you can also refer to the loader where each module is referenced, for example:

import style from 'style-loader!css-loader?modules!less-loader!./index.less'

In the file address/ index. Loaders can be added before less. Multiple loaders can be used! Split. At the same time, each loader can be added later? options as loader. This method of adding loaders is an inline type of loader. At the same time, a special tag prefix can be added to indicate what type of loader a specific model needs to use, which are as follows:

Symbol	variable	meaning
-!	noPreAutoLoaders	Do not pre load and ordinary loader
!	noAutoLoaders	No ordinary loader
!!	noPrePostAutoLoaders	Do not use front and back loaders and ordinary loaders, just inline loaders

For example, for the following:

import style from '-!style-loader!css-loader?modules!less-loader!./index.less'

For/ index. For the less module, you can't use the front ordinary loader configured in the configuration file. You can only use the rear and inline loaders to process this module.


Therefore, there are four types of loaders for processing modules: post, normal, inline and pre. There are three kinds of tags that can mark what type of loader a specific module uses. Next, see how it is implemented from the perspective of source code.

How does the loader work

Suppose there are the following files and rules:

const request = 'inline-loader1!inline-loader2!./src/index.js';
const rules = [
  {
    enforce: 'pre',
    test: /\.js$/,
    use: ['pre-loader1', 'pre-loader2'],
  },
  {
    enforce: 'normal',
    test: /\.js$/,
    use: ['normal-loader1', 'normal-loader2'],
  },
  {
    enforce: 'post',
    test: /\.js$/,
    use: ['post-loader1', 'post-loader2'],
  }
];

Here, request means that the module is/ src/index.js, and the module is processed by two inline loaders, async-loader1 and async-loader2. At the same time, there is also a rule in the webpack configuration file, including pre-loader1 and pre-loader2, and normal-loader1 and normal-loader2. Of course, when the enforce is not assigned, it is the default normal. There are post loaders post-loader1 and post-loader2.


First, we need to get these four loader s:

const preLoaders = [];
const normalLoaders = [];
const postLoaders = [];
const inlineLoaders = request.replace(/^-?!+/, "").replace(/!!+/g, '!').split('!');

for(let i = 0; i < rules.length; i++) {
  let rule = rules[i];
  if(rule.test.test(resource)) {
    if(rule.enforce === 'pre') {
      preLoaders.push(...rule.use);
    } else if(rule.enforce === 'post') {
      postLoaders.push(...rule.use);
    } else { // normal
      normalLoaders.push(...rule.use);
    }
  }
}

In order to get the inline loader, you need to use the referenced address! Split acquisition, but before that, you need to set -?! Special tag prefix is set to null, while for continuous! It also needs to be set to empty to avoid empty loaders. In this way, you can get ['async-loader1', 'async-loader2', '. / src/index.js'), and you can get the inline loaders. At the same time, you can get other loaders by looping through rules. So far, we have got four loaders. It is worth noting that the order of loaders in the reference address and rules is changed within the defined order.


Next, we need to get the order list of loaders. By default, that is, without special marks, loaders will be generated in the following order:

loaders = [
  ...postLoaders,
  ...inlineLoaders,
  ...normalLoaders,
  ...preLoaders,
 ];

By default, loaders are generated in the order of post inline normal pre and the order of each loader definition.


References with special tags also affect the contents of loaders:

if(request.startsWith('!')) { // Don't normal
  loaders = [
    ...postLoaders,
    ...inlineLoaders,
    ...preLoaders,
  ];
} else if(request.startsWith('-!')) { // Do not normal, pre
  loaders = [
    ...postLoaders,
    ...inlineLoaders
  ];
} else if(request.startsWith('!!')) { // Do not post, normal, pre
  loaders = [
    ...inlineLoaders,
  ];
} else { // post,inline,normal,pre
  loaders = [
    ...postLoaders,
    ...inlineLoaders,
    ...normalLoaders,
    ...preLoaders,
  ];
}

For the reference address, request is only with! At the beginning, normal type loaders are not required, but the order of other types of loaders remains. Similarly, -! Normal and pre loader are not required!! post, normal and pre loader are not required.


At this point, the loaders processing list of the reference file request has been obtained. Next, the loaders in the loader list need to be processed by the loader runner according to certain rules.

runLoaders({
  resource: path.join(__dirname, resource),
  loaders
}, (err, data) => {
  console.log(data);
});

The complete code for loader to obtain the loaders list is as follows:

const { runLoaders } = require('loader-runner');
const fs = require('fs');
const path = require('path');

const loadDir = path.resolve(__dirname,'loaders', 'runner');
const request = 'inline-loader1!inline-loader2!./src/index.js';

let preLoaders = [];
let normalLoaders = [];
let postLoaders = [];
let inlineLoaders = request.replace(/^-?!+/, "").replace(/!!+/g, '!').split('!');

const resource = inlineLoaders.pop();

const resolveLoader = loader => path.resolve(loadDir, loader);

const rules = [
  {
    enforce: 'pre',
    test: /\.js$/,
    use: ['pre-loader1', 'pre-loader2'],
  },
  {
    enforce: 'normal',
    test: /\.js$/,
    use: ['normal-loader1', 'normal-loader2'],
  },
  {
    enforce: 'post',
    test: /\.js$/,
    use: ['post-loader1', 'post-loader2'],
  }
];



for(let i = 0; i < rules.length; i++) {
  let rule = rules[i];
  if(rule.test.test(resource)) {
    if(rule.enforce === 'pre') {
      preLoaders.push(...rule.use);
    } else if(rule.enforce === 'post') {
      postLoaders.push(...rule.use);
    } else {
      normalLoaders.push(...rule.use);
    }
  }
}

preLoaders = preLoaders.map(resolveLoader);
normalLoaders = normalLoaders.map(resolveLoader);
inlineLoaders = inlineLoaders.map(resolveLoader);
postLoaders = postLoaders.map(resolveLoader);
let loaders = [];

if(request.startsWith('!')) { // Don't normal
  loaders = [
    ...postLoaders,
    ...inlineLoaders,
    ...preLoaders,
  ];
} else if(request.startsWith('-!')) { // Do not normal, pre
  loaders = [
    ...postLoaders,
    ...inlineLoaders
  ];
} else if(request.startsWith('!!')) { // Do not post, normal, pre
  loaders = [
    ...inlineLoaders,
  ];
} else { // post,inline,normal,pre
  loaders = [
    ...postLoaders,
    ...inlineLoaders,
    ...normalLoaders,
    ...preLoaders,
  ];
}

runLoaders({
  resource: path.join(__dirname, resource),
  loaders,
  readResource:fs.readFile.bind(fs)
}, (err, data) => {
  console.log(data);
});

To sum up, after webpack gets the reference address request of the module file, one step needs to be processed by the loader. First or take out the four loaders and assemble them into the loader execution list loaders according to post inline normal pre respectively. At the same time, the loader type can be filtered through special tags. However, the loader order remains the same. After obtaining the laoders, it will be handed over to the loader runner to further process the source file according to certain rules. Next, let's introduce the important loader runner in detail. First, explain the basic principles and concepts, and then implement a loader runner together.

Loader runner basic rules

Have you ever considered a problem, why do the loaders configured in the configuration file process the source files from right to left instead of from left to right? This is because when handling each loader, the loader runner will first execute the loader pitch method from left to right, and then execute its own loader method, which is called normal. As shown in the figure below:

As described in the previous section, there are loaders in a certain order in loaders, which will be executed by laoder runner. As shown in the following post-loader1 code, loaders can add a pitch method:

function loader(source){
   console.log('post-loader1 normal');
   return source+"[post-loader1]";
}
loader.pitch  = function(){
   console.log('post-pitch1 pitch');
}
module.exports = loader;

The loader method can be called normal method. This method mainly accepts the content of the source file as a parameter, and then returns the processed source file. In the example, the loader is mainly to add the [post-loader1] string after the source string, and then give the normal of the next loader as an input parameter. At the same time, a pitch method can also be added to the normal method, which is mainly used to do some preprocessing or interception before executing the loader.


Start to explain the figure below. The whole processing process is similar to the event bubbling mechanism of DOM. When calling loader runner, it will be executed in the order of laoders. First execute the pitch method in the loader. If the method does not return a value, continue to execute the next pitch until the normal of the last loader is executed. Then execute the normal method of the loader from right to left, and the return value of the previous loader is used as the input parameter of the next normal. However, if there is a loader's pitch return value in the middle, as shown in the red dotted line in the figure, the return value will be directly used as the input parameter of the previous loader normal, and then continue to execute. In this way, the sub will not parse the source code. For example, this scenario will be used in the cache.


For loaders, both normal and pitch can write synchronous code and asynchronous code. For synchronous code, you can directly return a value as the input parameter of the next loader. However, asynchronous is a little different. The specific code is as follows:

function loader(source) {
  const callback = this.async();
  setTimeout(() => {
    callback(null, source + "[async-loader1]");
  }, 3000);
}
loader.pitch = function () {
  const callback = this.async();
  console.log('async-loader1-pitch');
  callback(null);
}
module.exports = loader;

First, you need to call this Async function to declare that this is an asynchronous method and return a callback handle, which is used to execute subsequent processes after asynchronous execution. Callback provides the input parameters of err, and the next normal. The same is true for pitch. You can also daioyong this Async talks about changing a synchronous method to asynchronous.


So far, the loader execution process and synchronous and asynchronous loader are introduced. Next, we use the source code point of view to further understand the loader runner and the design based on the responsibility chain pattern.

Let's implement loader runner

The last section of loader runner also introduced that the overall process is similar to DOM event bubbling mechanism, scope chain, prototype chain, react event mechanism and koa onion model. Their core is based on the design pattern of responsibility chain pattern. Please refer to the responsibility chain model for details https://www.yuque.com/wmaoshu/blog/rcf95o For the responsibility chain mode, multiple objects can be processed uniformly, avoiding the coupling between the requestor and the receiving processor due to various types. In order to ensure that the request can be processed according to rules through multiple acceptances, a mechanism needs to be set to link the receiver, and then the requester will execute according to this link. Therefore, the important thing for the responsibility chain is: first, in order to ensure that the requested function has a single responsibility, it needs to be universal, that is, the function signature and return value should be consistent, and the ability to notify the chain to start the next one; Second, in order to ensure that the functional formula of the request is open and closed, a chain is needed to connect this process in series.

The options parameter will be provided in the loader code to be executed:

runLoaders({
  resource: path.join(__dirname, resource),
  loaders,
  readResource:fs.readFile.bind(fs)
}, (err, data) => {
  console.log(data);
});

runLoaders First parameter options The value is:
{
  resource: '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/src/index.js',
  loaders: [
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/post-loader1',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/post-loader2',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/inline-loader1',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/inline-loader2',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/normal-loader1',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/normal-loader2',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/pre-loader1',
    '/Users/mt/Documents/my-lib-learn/webpack/5.webpack-loader/loaders/runner/pre-loader2'
  ]
}

The first parameter options provided to the runLoader method includes the absolute address path resource of the resource and the list of loaders of the absolute address path of the loader.

For the runLoaders method, part of it is to create an execution context, and then call the iteratePitchingLoaders method to start the execution of laoder.

exports.runLoaders = function (options, callback) {
  createLoaderContext(options);
  let processOptions = {
    resourceBuffer: null, //Finally, we will put the Buffer result executed by the loader here
    readResource: options.readResource || readFile,
  }
  iteratePitchingLoaders(processOptions, loaderContext, function (err, result) {
    if (err) {
      return callback(err, {});
    }
    callback(null, {
      result,
      resourceBuffer: processOptions.resourceBuffer
    });
  });
};

Next, let's introduce the loader execution context and the loader object.

function parsePathQueryFragment(resource) { //resource =./src/index.js?name=wms#1
  let result = /^([^?#]*)(\?[^#]*)?(#.*)?$/.exec(resource);
  return {
    path: result[1], //Pathname/ src/index.js
    query: result[2], //   ?name=wms
    fragment: result[3] // #1
  }
};

//loader absolute path
// For example: / users / mt / documents / my lib learn / webpack / 5 webpack-loader/loaders/runner/post-loader1? {"presets":["/Users/anning/Desktop/webpack-demo/node_modules/@babel/preset-env"]}"
function createLoaderObject(loader) {
  let obj = {
    path: '', //Absolute path of current loader
    query: '', //Query parameters of current loader
    fragment: '', //Fragment of current loader
    normal: null, //The normal function of the current loader, that is, the loader function
    pitch: null, //pitch function of current loader
    raw: null, //Is it Buffer
    data: {}, //Custom object each loader will have a data custom object
    pitchExecuted: false, //The pitch function of the current loader has been executed, so it is unnecessary to execute it again
    normalExecuted: false //The normal function of the current loader has been executed, so it is unnecessary to execute it again
  }
  Object.defineProperty(obj, 'request', {
    get() {
      return obj.path + obj.query + obj.fragment;
    },
    set(value) {
      let splittedRequest = parsePathQueryFragment(value);
      obj.path = splittedRequest.path;
      obj.query = splittedRequest.query;
      obj.fragment = splittedRequest.fragment;
    }
  });
  obj.request = loader;
  return obj;
};

function loadLoader(loaderObject) {
  let normal = require(loaderObject.path);
  loaderObject.normal = normal;
  loaderObject.pitch = normal.pitch;
  loaderObject.raw = normal.raw;
};

function createLoaderContext(options) {
  // The absolute path of the resource to load
  const splittedResource = parsePathQueryFragment(options.resource || '');

  // Prepare loader object array
  loaders = (options.loaders || []).map(createLoaderObject);

  // The context object when the loader executes. This object will become the this pointer when the loader executes
  const loaderContext = {};

  loaderContext.context = path.dirname(splittedResource.path); // The directory of the resource to load
  loaderContext.loaderIndex = 0; //Currently processed loader index
  loaderContext.loaders = loaders; // loader collection
  loaderContext.resourcePath = splittedResource.path;//Resource absolute path
  loaderContext.resourceQuery = splittedResource.query;// query resolved in resource path
  loaderContext.resourceFragment = splittedResource.fragment;// Fragments resolved in the resource path
  loaderContext.async = null; //Is a method that can change the execution of the loader from synchronous to asynchronous
  loaderContext.callback = null; //Call the next loader

  // Full path to load resource
  Object.defineProperty(loaderContext, 'resource', {
    get() {
      return loaderContext.resourcePath + loaderContext.resourceQuery + loaderContext.resourceFragment;
    }
  });
  //request =loader1!loader2!loader3!./src/index.js
  Object.defineProperty(loaderContext, 'request', {
    get() {
      return loaderContext.loaders.map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //The remaining loaders are taken from the current next loader, plus resource
  Object.defineProperty(loaderContext, 'remainingRequest', {
    get() {
      return loaderContext.loaders.slice(loaderContext.loaderIndex + 1).map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //The current loader starts from the current loader and adds resource
  Object.defineProperty(loaderContext, 'currentRequest', {
    get() {
      return loaderContext.loaders.slice(loaderContext.loaderIndex).map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //Previous loader 
  Object.defineProperty(loaderContext, 'previousRequest', {
    get() {
      return loaderContext.loaders.slice(0, loaderContext.loaderIndex).map(l => l.request)
    }
  });
  //The query of the current loader. If options is configured in the configuration, use it. Otherwise, use the options in query
  Object.defineProperty(loaderContext, 'query', {
    get() {
      let loader = loaderContext.loaders[loaderContext.loaderIndex];
      return loader.options || loader.query;
    }
  });
  //The data of the current loader can be obtained in the pitch normal function
  Object.defineProperty(loaderContext, 'data', {
    get() {
      let loader = loaderContext.loaders[loaderContext.loaderIndex];
      return loader.data;
    }
  });
};

createLoaderContext is to generate a context for executing normal or pitch according to the passed in options, that is, this. In addition to some general parameters, async callback is more important. In the asynchronous loader described earlier, this This in async is the loaderContext. At the same time, loaderIndex is the global object. You can use loaderIndex to control which loader to execute and whether the process should go to the next step or the previous step. That is, each loader can guarantee that the responsibility can be single, and through this Callback to control whether to continue execution.

createLoaderObject is to create a loader object based on the absolute address of each loader, which contains functions such as normal, pitch, and so on. In addition, data is used to share data between normal and pitch in the same loader, as well as the flag bits of pitchExecuted and normalExecuted. The loadLoader function is used to load the loader, module Exports is normal, and then get pitch and raw.

Next, let's see how it is implemented. The execution is divided into three parts. One part is the iteratePitchingLoaders method, which is mainly used to control the flow between the pitch methods. The other is that iteratnormalloaders controls the flow of normal function in normal, and runSyncOrAsync that executes normal and pitch functions. When the pitch reaches the end, the processResource method is required to obtain the source file.


Here is the iteratePitchingLoaders Code:

function iteratePitchingLoaders(options, loaderContext, callback) {
  // After all the pitch es are processed, start to get the source code
  if (loaderContext.loaderIndex >= loaderContext.loaders.length) {
    return processResource(options, loaderContext, callback);
  }
  //Get the current loader
  const currentLoaderObject = loaderContext.loaders[loaderContext.loaderIndex];
  // The pitch has been processed. You need to process the pitch of the next loader
  if (currentLoaderObject.pitchExecuted) {
    loaderContext.loaderIndex++;
    return iteratePitchingLoaders(options, loaderContext, callback)
  }
  // Load laoder
  loadLoader(currentLoaderObject);
  let pitchFunction = currentLoaderObject.pitch;
  currentLoaderObject.pitchExecuted = true;
  if (!pitchFunction) {
    return iteratePitchingLoaders(options, loaderContext, callback);
  }

  runSyncOrAsync(
    pitchFunction, //The pitch function to execute
    loaderContext, //Context object
    //This is the array of parameters to be passed to pitchFunction
    [loaderContext.remainingRequest, loaderContext.previousRequest, loaderContext.data = {}],
    //Processing completed callbacks
    function (err, ...args) {
      if (args.length > 0) { //If args has a value, it indicates that this pitch has a return value
        loaderContext.loaderIndex--; //The index is minus 1 and begins to fall back
        iterateNormalLoaders(options, loaderContext, args, callback);
      } else { //If there is no return value, execute the pitch function of the next loader
        iteratePitchingLoaders(options, loaderContext, callback)
      }
    }
  );
};

Specifically, use loadLoader to load the loader processor, obtain the pitch function, set the pitch flag bit of laoder to true, indicating that the function begins to enter the processing stage. In the processing stage, if the obtained pitch function does not exist, iteratePitchingLoaders will be called directly to enter the next pitch execution. Otherwise, runSyncAsync will be called to execute the pitch function. After execution, regret according to the parameters passed in by callback. If there are parameters, loaderIndex will be subtracted to start executing the normal function of the previous loader, Otherwise, continue to execute the next pitch. After all the pitches are processed, start calling the processResource method to obtain the source code execution normal. The parameters passed in to execute the pitch include remainingRequest, previousRequest and data, mainly

function processResource(options, loaderContext, callback) {
  //Reset loaderIndex to loader length minus 1
  loaderContext.loaderIndex = loaderContext.loaders.length - 1;
  let resourcePath = loaderContext.resourcePath;
  //Call FS The readfile method reads the content of the resource
  options.readResource(resourcePath, function (err, buffer) {
    if (err) return callback(error);
    options.resourceBuffer = buffer; //resourceBuffer is the original content of the resource
    iterateNormalLoaders(options, loaderContext, [buffer], callback);
  });
}

After the processResource function obtains the resource source code, it starts to call normal. At this time, the loaderIndex should be increased by 1 and then decreased by 1 in iteratePitchingLoaders.


Next, we introduce the iteratenormallloaders function:

function iterateNormalLoaders(options, loaderContext, args, callback) {
  //If all normal loader s are executed
  if (loaderContext.loaderIndex < 0) {
    return callback(null, args);
  }
  let currentLoaderObject = loaderContext.loaders[loaderContext.loaderIndex];
  //If this normal has been executed, reduce the index by 1
  if (currentLoaderObject.normalExecuted) {
    loaderContext.loaderIndex--;
    return iterateNormalLoaders(options, loaderContext, args, callback)
  }
  let normalFn = currentLoaderObject.normal;
  currentLoaderObject.normalExecuted = true;

  runSyncOrAsync(normalFn, loaderContext, args, function (err) {
    if (err) return callback(err);
    let args = Array.prototype.slice.call(arguments, 1);
    iterateNormalLoaders(options, loaderContext, args, callback);
  });
}

The iteratenormallloaders method obtains the normal function, sets the normal flag bit true, starts calling runSyncOrAsync to execute the code, obtains the parameters after execution, and then continues to execute the next normal.

function runSyncOrAsync(fn, context, args, callback) {
  let isSync = true; //The default is synchronization
  let isDone = false; //Whether this function has been completed and executed. The default is false
  //Call context async this. Async can convert synchronization to asynchrony, which means that the code in the loader is asynchronous
  context.async = function () {
    isSync = false; //Change to asynchronous
    return innerCallback;
  }
  const innerCallback = context.callback = function () {
    isDone = true; //Indicates that the current function has been completed
    isSync = false; //Change to asynchronous
    callback.apply(null, arguments); //Execute callback
  }
  //The first time fn=pitch1, execute pitch1
  let result = fn.apply(context, args);
  //When pitch2 is executed, pitch1 is not executed yet
  if (isSync) {
    isDone = true;
    return callback(null, result);
  }
}

The of the pitch and normal functions of runSyncOrAsync substantive loader, if this is not called in the function Async will not execute context Aysnc, that is to say, if isSync is still true, the method will be executed directly, using loaderContext as this, and the return value as the result to guide the subsequent process. If it is an asynchronous function, you need to wait until the asynchronous execution of the function is completed, and then call callback to continue the subsequent execution. In other words, their execution is asynchronous and serial, not asynchronous and parallel.


Here is the complete code of laoder runner:

const fs = require('fs');
const path = require('path');
const readFile = fs.readFile.bind(fs);

function parsePathQueryFragment(resource) { //resource =./src/index.js?name=wms#1
  let result = /^([^?#]*)(\?[^#]*)?(#.*)?$/.exec(resource);
  return {
    path: result[1], //Pathname/ src/index.js
    query: result[2], //   ?name=wms
    fragment: result[3] // #1
  }
};

//loader absolute path
// For example: / users / mt / documents / my lib learn / webpack / 5 webpack-loader/loaders/runner/post-loader1? {"presets":["/Users/anning/Desktop/webpack-demo/node_modules/@babel/preset-env"]}"
function createLoaderObject(loader) {
  let obj = {
    path: '', //Absolute path of current loader
    query: '', //Query parameters of current loader
    fragment: '', //Fragment of current loader
    normal: null, //The normal function of the current loader, that is, the loader function
    pitch: null, //pitch function of current loader
    raw: null, //Is it Buffer
    data: {}, //Custom object each loader will have a data custom object
    pitchExecuted: false, //The pitch function of the current loader has been executed, so it is unnecessary to execute it again
    normalExecuted: false //The normal function of the current loader has been executed, so it is unnecessary to execute it again
  }
  Object.defineProperty(obj, 'request', {
    get() {
      return obj.path + obj.query + obj.fragment;
    },
    set(value) {
      let splittedRequest = parsePathQueryFragment(value);
      obj.path = splittedRequest.path;
      obj.query = splittedRequest.query;
      obj.fragment = splittedRequest.fragment;
    }
  });
  obj.request = loader;
  return obj;
};

function loadLoader(loaderObject) {
  let normal = require(loaderObject.path);
  loaderObject.normal = normal;
  loaderObject.pitch = normal.pitch;
  loaderObject.raw = normal.raw;
};

function createLoaderContext(options) {
  // The absolute path of the resource to load
  const splittedResource = parsePathQueryFragment(options.resource || '');

  // Prepare loader object array
  loaders = (options.loaders || []).map(createLoaderObject);

  // The context object when the loader executes. This object will become the this pointer when the loader executes
  const loaderContext = {};

  loaderContext.context = path.dirname(splittedResource.path); // The directory of the resource to load
  loaderContext.loaderIndex = 0; //Currently processed loader index
  loaderContext.loaders = loaders; // loader collection
  loaderContext.resourcePath = splittedResource.path;//Resource absolute path
  loaderContext.resourceQuery = splittedResource.query;// query resolved in resource path
  loaderContext.resourceFragment = splittedResource.fragment;// Fragments resolved in the resource path
  loaderContext.async = null; //Is a method that can change the execution of the loader from synchronous to asynchronous
  loaderContext.callback = null; //Call the next loader

  // Full path to load resource
  Object.defineProperty(loaderContext, 'resource', {
    get() {
      return loaderContext.resourcePath + loaderContext.resourceQuery + loaderContext.resourceFragment;
    }
  });
  //request =loader1!loader2!loader3!./src/index.js
  Object.defineProperty(loaderContext, 'request', {
    get() {
      return loaderContext.loaders.map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //The remaining loaders are taken from the current next loader, plus resource
  Object.defineProperty(loaderContext, 'remainingRequest', {
    get() {
      return loaderContext.loaders.slice(loaderContext.loaderIndex + 1).map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //The current loader starts from the current loader and adds resource
  Object.defineProperty(loaderContext, 'currentRequest', {
    get() {
      return loaderContext.loaders.slice(loaderContext.loaderIndex).map(l => l.request).concat(loaderContext.resource).join('!')
    }
  });
  //Previous loader 
  Object.defineProperty(loaderContext, 'previousRequest', {
    get() {
      return loaderContext.loaders.slice(0, loaderContext.loaderIndex).map(l => l.request)
    }
  });
  //The query of the current loader. If options is configured in the configuration, use it. Otherwise, use the options in query
  Object.defineProperty(loaderContext, 'query', {
    get() {
      let loader = loaderContext.loaders[loaderContext.loaderIndex];
      return loader.options || loader.query;
    }
  });
  //The data of the current loader can be obtained in the pitch normal function
  Object.defineProperty(loaderContext, 'data', {
    get() {
      let loader = loaderContext.loaders[loaderContext.loaderIndex];
      return loader.data;
    }
  });
};

function convertArgs(args, raw) {
  // If this loader needs buffer, args[0] is not, it needs to be converted to buffer
  if (raw && !Buffer.isBuffer(args[0])) {
    args[0] = Buffer.from(args[0], 'utf8');
  } else if (!raw && Buffer.isBuffer(args[0])) {
    args[0] = args[0].toString('utf8');
  }
};

function runSyncOrAsync(fn, context, args, callback) {
  let isSync = true; //The default is synchronization
  let isDone = false; //Whether this function has been completed and executed. The default is false
  //Call context async this. Async can convert synchronization to asynchrony, which means that the code in the loader is asynchronous
  context.async = function () {
    isSync = false; //Change to asynchronous
    return innerCallback;
  }
  const innerCallback = context.callback = function () {
    isDone = true; //Indicates that the current function has been completed
    isSync = false; //Change to asynchronous
    callback.apply(null, arguments); //Execute callback
  }
  //The first time fn=pitch1, execute pitch1
  let result = fn.apply(context, args);
  //When pitch2 is executed, pitch1 is not executed yet
  if (isSync) {
    isDone = true;
    return callback(null, result);
  }
}

function processResource(options, loaderContext, callback) {
  //Reset loaderIndex to loader length minus 1
  loaderContext.loaderIndex = loaderContext.loaders.length - 1;
  let resourcePath = loaderContext.resourcePath;
  //Call FS The readfile method reads the content of the resource
  options.readResource(resourcePath, function (err, buffer) {
    if (err) return callback(error);
    options.resourceBuffer = buffer; //resourceBuffer is the original content of the resource
    iterateNormalLoaders(options, loaderContext, [buffer], callback);
  });
}

function iterateNormalLoaders(options, loaderContext, args, callback) {
  //If all normal loader s are executed
  if (loaderContext.loaderIndex < 0) {
    return callback(null, args);
  }
  let currentLoaderObject = loaderContext.loaders[loaderContext.loaderIndex];
  //If this normal has been executed, reduce the index by 1
  if (currentLoaderObject.normalExecuted) {
    loaderContext.loaderIndex--;
    return iterateNormalLoaders(options, loaderContext, args, callback)
  }
  let normalFn = currentLoaderObject.normal;
  currentLoaderObject.normalExecuted = true;
  convertArgs(args, currentLoaderObject.raw);
  runSyncOrAsync(normalFn, loaderContext, args, function (err) {
    if (err) return callback(err);
    let args = Array.prototype.slice.call(arguments, 1);
    iterateNormalLoaders(options, loaderContext, args, callback);
  });
}

function iteratePitchingLoaders(options, loaderContext, callback) {
  // After all the pitch es are processed, start to get the source code
  if (loaderContext.loaderIndex >= loaderContext.loaders.length) {
    return processResource(options, loaderContext, callback);
  }
  //Get the current loader
  const currentLoaderObject = loaderContext.loaders[loaderContext.loaderIndex];
  // The pitch has been processed. You need to process the pitch of the next loader
  if (currentLoaderObject.pitchExecuted) {
    loaderContext.loaderIndex++;
    return iteratePitchingLoaders(options, loaderContext, callback)
  }
  // Load laoder
  loadLoader(currentLoaderObject);
  let pitchFunction = currentLoaderObject.pitch;
  currentLoaderObject.pitchExecuted = true;
  if (!pitchFunction) {
    return iteratePitchingLoaders(options, loaderContext, callback);
  }

  runSyncOrAsync(
    pitchFunction, //The pitch function to execute
    loaderContext, //Context object
    //This is the array of parameters to be passed to pitchFunction
    [loaderContext.remainingRequest, loaderContext.previousRequest, loaderContext.data = {}],
    //Processing completed callbacks
    function (err, ...args) {
      if (args.length > 0) { //If args has a value, it indicates that this pitch has a return value
        loaderContext.loaderIndex--; //The index is minus 1 and begins to fall back
        iterateNormalLoaders(options, loaderContext, args, callback);
      } else { //If there is no return value, execute the pitch function of the next loader
        iteratePitchingLoaders(options, loaderContext, callback)
      }
    }
  );
};


exports.runLoaders = function (options, callback) {
  createLoaderContext(options);
  let processOptions = {
    resourceBuffer: null, //Finally, we will put the Buffer result executed by the loader here
    readResource: options.readResource || readFile,
  }
  iteratePitchingLoaders(processOptions, loaderContext, function (err, result) {
    if (err) {
      return callback(err, {});
    }
    callback(null, {
      result,
      resourceBuffer: processOptions.resourceBuffer
    });
  });
};

So far, we have learned what the loader is, how to configure it in the webpack, and how to implement the loader principle. We will further understand it from the perspective of the loader runner source code. Next is the simple implementation of CSS loader and style loader.

Start writing loader

The following is a simple implementation of CSS loader:

let postcss = require('postcss');
let loaderUtils  = require('loader-utils');
let Tokenizer = require('css-selector-tokenizer');
/**
 * postcss It is used to deal with CSS and is also based on CSS abstract syntax tree
 */
function loader(cssString){
    const cssPlugin = (options)=>{
        return (cssRoot)=>{
            //Traverse the syntax tree and find all import statements
            cssRoot.walkAtRules(/^import$/i,rule=>{
                rule.remove();//Delete this import
                let imp = rule.params.slice(1,-1);
                options.imports.push(imp);
            });
            cssRoot.walkDecls(decl=>{
                let values = Tokenizer.parseValues(decl.value);
                values.nodes.forEach(function(value){
                    value.nodes.forEach(item=>{
                        if(item.type === 'url'){
                            item.url = "`+require("+loaderUtils.stringifyRequest(this,item.url)+").default+`";
                            console.log('====item',item);
                        }
                    });
                });
                decl.value = Tokenizer.stringifyValues(values);
            });
        }
    }
    let callback = this.async();
    let options = {imports:[]};//["./global.css"]
    //The source code will go through the plug-ins of the pipeline 
    let pipeLine = postcss([cssPlugin(options)]);
    pipeLine.process(cssString).then(result=>{
        let importCSS = options.imports.map(url=>{
            return "`+require("+loaderUtils.stringifyRequest(this,"!!css-loader2!"+url)+")+`";
        }).join('\r\n');
        let output = "module.exports = `"+importCSS+"\n"+result.css+"`";
        output=output.replace(/\\"/g,'"');
        callback(null,output);
    });
}
module.exports = loader;

style-loader:

let loaderUtils = require('loader-utils');
function loader(source){

};

loader.pitch = function(remainingRequest,previousRequest,data) {
  let script = `
    let style = document.createElement('style');
    style.innerHTML = require(${loaderUtils.stringifyRequest(this,"!!"+remainingRequest)});
    document.head.appendChild(style);
  `;
  return script;
};

module.exports = loader;