Node.js multi process

Posted by lunas on Sat, 05 Mar 2022 17:31:01 +0100

1, Node JS multi process

We all know node JS runs in a single thread mode, but it uses event driven to handle concurrency, which helps us create multiple sub processes on a multi-core cpu system to improve performance.
Each child process always has three flow objects: child stdin, child. Stdout and child stderr. They may share the stdio flow of the parent process, or they can be independent diverted flow objects.
Node provides child_process module to create a sub process. The methods are as follows:

  • exec - child_process.exec uses the subprocess to execute commands, caches the output of the subprocess, and returns the output of the subprocess in the form of callback function parameters.
  • spawn - child_process.spawn creates a new process using the specified command line parameters.
  • fork - child_process.fork is a special form of spawn(), which is used to run modules in subprocesses. For example, fork('. / son.js') is equivalent to spawn(' node ', ['. / son.js'). Unlike the spawn method, fork will establish a communication pipeline between the parent process and the child process for communication between processes.

2, exec() method

child_process.exec uses the subprocess to execute commands, caches the output of the subprocess, and returns the output of the subprocess in the form of callback function parameters.
(1) The syntax is as follows:

  • child_process.exec(command[, options], callback)

(1) The parameters are described as follows:

  • Command: string, the command to be run, and the parameters are separated by spaces
  • options: object, which can be:
    • cwd, string, current working directory of child process
    • env, object environment variable key value pair
    • Encoding, string, character encoding (default: 'utf8')
    • Shell, string, shell to execute command (default: in UNIX, / bin/sh, in Windows, / s /c switch in UNIX, or / s /c in Windows. In Windows, command line parsing should be compatible with cmd.exe)
    • Timeout, number, timeout (default: 0)
    • maxBuffer, a number, is the maximum buffer (binary) allowed in stdout or stderr. If it is exceeded, the child process will be killed (default: 200 * 1024)
    • killSignal, string, end signal (default: 'SIGTERM')
    • uid, number, set the ID of the user process
    • gid, number, set the ID of the process group
  • Callback: callback function, including three parameters: error, stdout and stderr.
  • The exec() method returns the largest buffer and waits for the end of the process to return the contents of the buffer at one time.
// Example: create two JS files support JS and master js. 
// support.js file code:
console.log("process " + process.argv[2] + " Execution." );

// master.js file code:
const fs = require('fs');
const child_process = require('child_process');
 
for(var i=0; i<3; i++) {
    var workerProcess = child_process.exec('node support.js '+i, function (error, stdout, stderr) {
        if (error) {
            console.log(error.stack);
            console.log('Error code: '+error.code);
            console.log('Signal received: '+error.signal);
        }
        console.log('stdout: ' + stdout);
        console.log('stderr: ' + stderr);
    });
 
    workerProcess.on('exit', function (code) {
        console.log('Subprocess exited, exit code '+code);
    });
}
// Execute the above code and the output result is:
Subprocess exited with exit code 0
stdout: Process 1 executes.

stderr: 
Subprocess exited with exit code 0
stdout: Process 0 is executing.

stderr: 
Subprocess exited with exit code 0
stdout: Process 2 executes.

stderr: 

3, spawn() method

child_process.spawn uses the specified command line parameters to create a new process. The syntax format is as follows:

  • child_process.spawn(command[, args][, options])

(1) The parameters are described as follows:

  • Command: the command to be run
  • args: Array string parameter Array
  • options Object
    • cwd String current working directory of child process
    • env Object environment variable key value pair
    • stdio Array|String stdio configuration of child process
    • The detached Boolean subprocess will become the leader of the process group
    • uid Number sets the ID of the user process
    • gid Number sets the ID of the process group
  • The spawn() method returns the stream (stdout & stderr), which is used when the process returns a large amount of data. Once the process starts executing, spawn() starts receiving responses.
// example
// Create two JS files support JS and master js. 
// support.js file code:
console.log("process " + process.argv[2] + " Execution." );

// master.js file code:
const fs = require('fs');
const child_process = require('child_process');
 
for(var i=0; i<3; i++) {
   var workerProcess = child_process.spawn('node', ['support.js', i]);
 
   workerProcess.stdout.on('data', function (data) {
      console.log('stdout: ' + data);
   });
 
   workerProcess.stderr.on('data', function (data) {
      console.log('stderr: ' + data);
   });
 
   workerProcess.on('close', function (code) {
      console.log('Subprocess exited, exit code '+code);
   });
}

// Execute the above code and the output result is:
Subprocess exited with exit code 0
stdout: Process 1 executes.

Subprocess exited with exit code 0
stdout: Process 2 executes.

Subprocess exited with exit code 0

4, fork method

child_process.fork is a special form of spawn() method, which is used to create a process. The syntax format is as follows:

  • child_process.fork(modulePath[, args][, options])

The parameters are described as follows:

  • modulePath: String, the module to be run in the child process
  • args: Array string parameter Array
  • options: Object
    • cwd String current working directory of child process
    • env Object environment variable key value pair
    • execPath String creates the executable of the child process
    • execArgv Array the string parameter array of the executable file of the subprocess (default: process.execArgv)
    • If they are true, they will be inherited from the parent process to the child process. Otherwise, they will inherit from the parent process to the child process. (default: false)
    • uid Number sets the ID of the user process
    • gid Number sets the ID of the process group

In addition to all the methods that own the ChildProcess instance, the returned object also has a built-in communication channel.

// example
// Create two JS files support JS and master js. 
// support.js file code:
console.log("process " + process.argv[2] + " Execution." );

// master.js file code:
const fs = require('fs');
const child_process = require('child_process');
 
for(var i=0; i<3; i++) {
   var worker_process = child_process.fork("support.js", [i]);    
 
   worker_process.on('close', function (code) {
      console.log('Subprocess exited, exit code ' + code);
   });
}

// Execute the above code and the output result is:
Process 0 is executing.
Subprocess exited with exit code 0
 Process 1 executes.
Subprocess exited with exit code 0
 Process 2 executes.
Subprocess exited with exit code 0

Topics: node.js Back-end