Use asynchronous/wait in forEach loop

Posted by Garath531 on Tue, 17 Dec 2019 03:16:20 +0100

Is there any problem using async / await in the forEach loop?I'm trying to traverse the file array and await the contents of each file.

import fs from 'fs-promise'

async function printFiles () {
  const files = await getFilePaths() // Assume this works fine

  files.forEach(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  })
}

printFiles()

This code does work, but could it be a problem?I asked someone to tell me that you should not use async / await in such a higher-order function, so I just wanted to ask if there was a problem.

#1st floor

Make sure the code works, but I'm sure it won't do what you expect.It simply triggers multiple asynchronous calls, but the printFiles function does return immediately after printFiles.

If forEach reads files sequentially, forEach cannot actually be used.Just use the modern for...of loop in which await will work as expected:

async function printFiles () {
  const files = await getFilePaths();

  for (const file of files) {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  }
}

If you want to read files in parallel, you can't actually use forEach.Each async callback function call does return a Promise, but you throw them away instead of waiting for them.With just map, you can wait for Promise.all to get an array of promises:

async function printFiles () {
  const files = await getFilePaths();

  await Promise.all(files.map(async (file) => {
    const contents = await fs.readFile(file, 'utf8')
    console.log(contents)
  }));
}

#2nd floor

On npm p-iteration Modules implement the Array iteration method, so they can be used with async / await very easily.

An example of your case:

const { forEach } = require('p-iteration');
const fs = require('fs-promise');

(async function printFiles () {
  const files = await getFilePaths();

  await forEach(files, async (file) => {
    const contents = await fs.readFile(file, 'utf8');
    console.log(contents);
  });
})();

#3rd floor

Both of the above solutions work, but Antonio does it with less code, which is how it helps me parse data from a database, from several different sub-references, then push them all into an array and solve them as promise s, and do it:

Promise.all(PacksList.map((pack)=>{
    return fireBaseRef.child(pack.folderPath).once('value',(snap)=>{
        snap.forEach( childSnap => {
            const file = childSnap.val()
            file.id = childSnap.key;
            allItems.push( file )
        })
    })
})).then(()=>store.dispatch( actions.allMockupItems(allItems)))

#4th floor

It is easy to pop up several methods in a file that will process asynchronous data in a serialized order and provide a more traditional style to your code.For example:

module.exports = function () {
  var self = this;

  this.each = async (items, fn) => {
    if (items && items.length) {
      await Promise.all(
        items.map(async (item) => {
          await fn(item);
        }));
    }
  };

  this.reduce = async (items, fn, initialValue) => {
    await self.each(
      items, async (item) => {
        initialValue = await fn(initialValue, item);
      });
    return initialValue;
  };
};

Now, assuming you save it in'. /myAsync.js', you can do something similar in adjacent files:

...
/* your server setup here */
...
var MyAsync = require('./myAsync');
var Cat = require('./models/Cat');
var Doje = require('./models/Doje');
var example = async () => {
  var myAsync = new MyAsync();
  var doje = await Doje.findOne({ name: 'Doje', noises: [] }).save();
  var cleanParams = [];

  // FOR EACH EXAMPLE
  await myAsync.each(['bork', 'concern', 'heck'], 
    async (elem) => {
      if (elem !== 'heck') {
        await doje.update({ $push: { 'noises': elem }});
      }
    });

  var cat = await Cat.findOne({ name: 'Nyan' });

  // REDUCE EXAMPLE
  var friendsOfNyanCat = await myAsync.reduce(cat.friends,
    async (catArray, friendId) => {
      var friend = await Friend.findById(friendId);
      if (friend.name !== 'Long cat') {
        catArray.push(friend.name);
      }
    }, []);
  // Assuming Long Cat was a friend of Nyan Cat...
  assert(friendsOfNyanCat.length === (cat.friends.length - 1));
}

#5th floor

An important warning is that the await + for.. of and forEach + async methods of the method actually have different effects.

In a true for loop, await ensures that all asynchronous calls are executed one after the other.And forEach + async triggers all Promise s at once, which is faster but sometimes confusing (if you do some database queries or access some volume-limited Web services and don't want to trigger 100,000 calls at a time).

If you don't use async/await and want to make sure the files are read one after another, you can also use reduce + promise (less elegant).

files.reduce((lastPromise, file) => 
 lastPromise.then(() => 
   fs.readFile(file, 'utf8')
 ), Promise.resolve()
)

Alternatively, you can create a forEachAsync to help, but basically you can use the same for loop base.

Array.prototype.forEachAsync = async function(cb){
    for(let x of this){
        await cb(x);
    }
}

Topics: Database less npm