Using filesystem in node.js with async / await

Question

I would like to use async/await with some filesystem operations. Normally async/await works fine because I use babel-plugin-syntax-async-functions.

But with this code I run into the if case where names is undefined:

import fs from 'fs';

async function myF() {
  let names;
  try {
    names = await fs.readdir('path/to/dir');
  } catch (e) {
    console.log('e', e);
  }
  if (names === undefined) {
    console.log('undefined');
  } else {
    console.log('First Name', names[0]);
  }
}

myF();

When I rebuild the code into the callback hell version everything is OK and I get the filenames. Thanks for your hints.

Solution 1

Native support for async await fs functions since Node 11

Since Node.JS 11.0.0 (stable), and version 10.0.0 (experimental), you have access to file system methods that are already promisify'd and you can use them with try catch exception handling rather than checking if the callback's returned value contains an error.

The API is very clean and elegant! Simply use .promises member of fs object:

import fs from 'fs';
const fsPromises = fs.promises;

async function listDir() {
  try {
    return fsPromises.readdir('path/to/dir');
  } catch (err) {
    console.error('Error occured while reading directory!', err);
  }
}

listDir();

Solution 2

Starting with node 8.0.0, you can use this:

const fs = require('fs');
const util = require('util');

const readdir = util.promisify(fs.readdir);

async function myF() {
  let names;
  try {
    names = await readdir('path/to/dir');
  } catch (err) {
    console.log(err);
  }
  if (names === undefined) {
    console.log('undefined');
  } else {
    console.log('First Name', names[0]);
  }
}

myF();

See https://nodejs.org/dist/latest-v8.x/docs/api/util.html#util_util_promisify_original

Solution 3

Node.js 8.0.0

Native async / await

Promisify

From this version, you can use native Node.js function from util library.

const fs = require('fs')
const { promisify } = require('util')

const readFileAsync = promisify(fs.readFile)
const writeFileAsync = promisify(fs.writeFile)

const run = async () => {
  const res = await readFileAsync('./data.json')
  console.log(res)
}

run()

---

Promise Wrapping

const fs = require('fs')

const readFile = (path, opts = 'utf8') =>
  new Promise((resolve, reject) => {
    fs.readFile(path, opts, (err, data) => {
      if (err) reject(err)
      else resolve(data)
    })
  })

const writeFile = (path, data, opts = 'utf8') =>
  new Promise((resolve, reject) => {
    fs.writeFile(path, data, opts, (err) => {
      if (err) reject(err)
      else resolve()
    })
  })

module.exports = {
  readFile,
  writeFile
}

...


// in some file, with imported functions above
// in async block
const run = async () => {
  const res = await readFile('./data.json')
  console.log(res)
}

run()

Advice

Always use try..catch for await blocks, if you don't want to rethrow exception upper.

Solution 4

As of v10.0, you can use fs.Promises

Example using readdir

const { promises: fs } = require("fs");

async function myF() {
    let names;
    try {
        names = await fs.readdir("path/to/dir");
    } catch (e) {
        console.log("e", e);
    }
    if (names === undefined) {
        console.log("undefined");
    } else {
        console.log("First Name", names[0]);
    }
}

myF();

Example using readFile

const { promises: fs } = require("fs");

async function getContent(filePath, encoding = "utf-8") {
    if (!filePath) {
        throw new Error("filePath required");
    }

    return fs.readFile(filePath, { encoding });
}

(async () => {
    const content = await getContent("./package.json");

    console.log(content);
})();

Solution 5

You might produce the wrong behavior because the File-Api fs.readdir does not return a promise. It only takes a callback. If you want to go with the async-await syntax you could 'promisify' the function like this:

function readdirAsync(path) {
  return new Promise(function (resolve, reject) {
    fs.readdir(path, function (error, result) {
      if (error) {
        reject(error);
      } else {
        resolve(result);
      }
    });
  });
}

and call it instead:

names = await readdirAsync('path/to/dir');

View more solutions

Comments

[PirateApp]

Downloading a library for this is pure overkill, dependency bloating is something that the community should be strongly against, infact a new npmjs should come into making that only has libs with 0 dependencies

[makerj]

In node v8.9.4, got a SyntaxError: Unexpected token import error message. does node8 supports import token by default?

[Josh Sandlin]

@makerj he's using the new import syntax. It currently requires some transpiling. Would be ok to also use const fs = require('fs') or const { promisify } = require('util')

[Qasim]

Noob question, but what's the {err, names} = function syntax called?

[jaredkwright]

@Qasim it is called destructuring assignment.

[Alexander Zeitler]

@jaredkwright The destructuring call results in Unexpected token = using Node 10.1.0.

[jaredkwright]

@AlexanderZeitler That may be true. I haven't looked to see if that is actually a correct use of destructuring. In the case of async await I think you would just do names = await readdir('path/to/dir'); and if there is an err handle it in the catch block. Either way, the name of the syntax is destructuring assignment which was just in response to Qasim's question.

[Amiga500]

This is strange. I am getting SyntaxError: await is only valid in async function... crying in rage.

[dimpiax]

@VedranMaricevic. look at comments, await must be always in async block :)

[Jayraj]

@VedranMaricevic. You need to call that const res = await readFile('data.json') console.log(res) in some async function

[TheHanna]

This API is stable as of version 11.x per the File System documentation on the Node.js site

[Anthony]

This is so ugly, why do the only answers to this question throw errors on every version of node? Is it that hard to add an extra line to destructure longform?

[DavidP]

Works great, but important to note the open issue regarding the ExperimentalWarning: The fs.promises API is experimental warning: github.com/pnpm/pnpm/issues/1178

[Dan Starns]

@DavidP what version of node are you using? 12 and above works fine

[DavidP]

Yes! Absolutely correct - I neglected to state version I am on: v10.15.3 - it's possible to suppress the message. However, with the issue still open I thought it worth mentioning.

[Dan Starns]

@DavidP I mean it is worth a mention don't get me wrong, but node 12 is in LTS now so it's not a Biggie.

[oldboy]

promise wrapping fs.promises and using it with async/await is so confusing to me

[dimpiax]

@PrimitiveNom Promise can be used in traditional way within then, catch etc. Where are async/await is modern behavior flow.

[oldboy]

how exactly do you use this with, say, readFile? im new to this whole promises thing, and all i want to do is have a function getContent that i can call and await in various parts throughout my script, yet this is proving very confusing

[Dan Starns]

@PrimitiveNom you could use fs.promises.readFile I have added a code snippet to my answer.

[TOPKAT]

@DanStarns if you don't return await your promise, the catch block is of no use...I thinj it is sometimes a good practice to await before returning

[Dan Starns]

@538ROMEO just looked into this & your right. Thanks for pointing it out.

[Jeevan Takhar]

Documentation for these alternative methods: nodejs.org/api/fs.html#fs_fs_promises_api

[Paula Fleck]

I'm getting a weird response like this... Buffer(18524) [60, 115, 99, 114, 105, 112, 116, 32, 116, 110, 116, 45, 108, 105, 98, 62, 13, 10, 32, 32, 32, 32, 47, 42, 42, 13, 10, 32, 32, 32, 32, 32, 42, 32, 67, 111, 112, 121, 114, 105, 103, 104, 116, 32, 63, 32, 50, 48, 50, 48, 32, 68, 101, 115, 105, 103, 110, 32, 65 …]

[Jan Aagaard]

In TypeScript (and modern JavaScript?) you can write import { promises as fs } from "fs";.

[Dan Starns]

@JanAagaard yeah, you can. There are 2 answers below with import.

[Muhammed Bera Koç]

Use Buffer.toString method.

[Jake Wilson]

How come you don't have to put await before fsPromises.readdir(...)? I thought async/await have to always be used together. In this case the await seems to cause problems.

[Rana Ghosh]

i like doing import { promises as fs } from 'fs';