Why is WSL extremely slow when compared with native Windows NPM/Yarn processing?

Question

I am working with WSL a lot lately because I need some native UNIX tools (and emulators aren't good enough). I noticed that the speed difference when working with NPM/Yarn is incredible.

I conducted a simple test that confirmed my feelings. The test was running npx create-react-app my-test-app and the WSL result was Done in 287.56s. while GitBash finished with Done in 10.46s..

This is not the whole picture, because the perceived time was higher in both cases, but even based on that - there is a big issue somewhere. I just don't know where. The project I'm working on uses tens of libraries and changing even one of them takes minutes instead of seconds.

Is this something that I can fix? If so - where to look for clues?

Additional info:

• my processor: Processor AMD Ryzen 7 5800H with Radeon Graphics, 3201 Mhz, 8 Core(s), 16 Logical Processors

• I'm running Windows 11 with all the latest updates to both the system and the WSL. The chosen system is Ubuntu 20.04

• I've seen some questions that are somewhat similar like 'npm install' extremely slow on Windows, but they don't touch WSL at all (and my pure Windows NPM works fast).

• the issue is not limited to NPM, it's also for Yarn

• another problem that I'm getting is that file watching is not happening (I need to restart the server with every change). In some applications I don't get any errors, sometimes I get the following:

  Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/DumpStack.log.tmp'
  Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/hiberfil.sys'
  Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'
  Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/swapfile.sys'
  

• npm start in an empty (freshly initialized) create-react-app takes ages to render something in the browser in WSL and when executed from GitBash - I can see stuff in 2-4 seconds

• it is possible that's it's purely a WSL problem, but it just hurts the most when using NPM/Yarn

Solution 1

Since you mention executing the same files (with proper performance) from within Git Bash, I'm going to make an assumption here. Correct me if I'm wrong on this, and I'll delete the answer and look for another possibility.

This would be explained (and expected) if your files are stored on /mnt/c (a.k.a. C:, or /C under Git Bash) or any other Windows drive, as they would likely need to be to be accessed by Git Bash.

WSL2 uses the 9P protocol to access Windows drives, and it is currently known to be very slow when compared to:

• Native NTFS (obviously)
• The ext4 filesystem on the virtual disk used by WSL2
• And even the performance of WSL1 with Windows drives

I've seen a git clone of a large repo (the WSL2 Linux kernel Github) take 8 minutes on WSL2 on a Windows drive, but only seconds on the root filesystem.

Two possibilities:

• If possible (and it is for most Node projects), convert your WSL to version 1 with wsl --set-version <distroname> 1. I always recommend making a backup with wsl --export first.

  And since you are making a backup anyway, you may as well just create a copy of the instance by wsl --importing your backup as --version 1 (as the last argument). WSL1 and WSL2 both have their uses, and you may find it helpful to keep both around.

  See this answer for more details on the exact syntax..

• Or just move the project over to somewhere under the WSL root, such as /home/username/src/.

Solution 2

Building on @notthedr01ds's reponse.

If you look at Microsoft's Comparing WSL 1 and WSL 2 the 'Performance across the OS file systems' is explicitly worse in WSL2.

My case fell into Exceptions for using WSL 1 rather than WSL 2

• Your project files must be stored in the Windows file system. WSL 1 offers faster access to files mounted from Windows.
  • If you will be using your WSL Linux distribution to access project files on the Windows file system, and these files cannot be stored on the Linux file system, you will achieve faster performance across the OS files systems by using WSL 1.

This means I needed to swap to version 1.

wsl --set-version Ubuntu 1
Conversion in progress, this may take a few minutes...
Conversion complete.

Test before

>time git status
...
real    0m6.436s
user    0m0.055s
sys     0m36.380s

Test after

> time git status
...
real    0m0.126s
user    0m0.016s
sys     0m0.641s

Comments

[WrRaThY]

hey first of all - thank you very much for a comprehensive answer. and yes - all my files are stored under /mnt/c. I tried your 2nd solution - it worked as fast as expected in the terminal, but the IDE froze when I first loaded the create-react-app project [IntelliJ Ultimate]. interestingly enough, the second run was successful - I'll test more with my actual project. When it comes to switching to WSL1 - I vaguely remember that I needed to upgrade because WSL1 was missing some features that WSL2 had (I'm running Rust with the BPF compiler for Solana dev), so I can't do that

[WrRaThY]

I'll move with my real project to /home/username/... and see how the IDE handles the whole thing. if it works - I'll be happy to accept your answer, for now I'll +1 it (as I believe it will probably help someone in the future - please don't delete it) disclaimer: yes, I use Rust as my main programming language for this project, but node is used for E2E testing and therefore it has both - hence I used npm as a more widely known example to depict the problem. also - some npm libs (for testing solana) work only under *nix systems

[WrRaThY]

the IDE is considerably slower and freezes with more instances open (never happened before), but I guess it's fast enough and I'll just go with that solution for now. thank you :) PS. do you happen to have any information on whether this issue might be addressed in the future or not? is there any github issue opened for it?

[NotTheDr01ds]

@WrRaThY Here's the main issue for tracking, and while I've read statements from the Microsoft team that they aren't happy with the performance, I'm unaware of a root cause being mentioned (but I haven't read through all 350+ comments on the issue) other than perhaps the lack of caching support in 9P.

[NotTheDr01ds]

@WrRaThY Also, I didn't notice the "watching" part of your question earlier, but yes, that's also due to 9P. See this question/answer for more info on that. Note that watching seems to work if you run the app from within VSCode, though.

[WrRaThY]

thanks for all the support! I'm guessing that running from vscode = running from native windows filesystem, so this is probably why. I'll follow up on those links if it makes sense. cheers!

[nullromo]

I can confirm, I just moved my whole project from /mnt/c/... to /home/... and the speed increase is astonishing.

[Marian Klühspies]

My project is stored under /home/user/workspace/myproject and I'm running into performance issues. Sometimes it seems yarn install is stuck for a while until it processes further. Accessing the project with WebStorm under Windows 11 has also a lot of freezes.

[vdegenne]

For development WSL2 is a big nay. And not only it's super slow I had issues making file change detection works properly. Going back to v1..

[NotTheDr01ds]

@vdegenne WSL1 is a good option, but also remember that having the files on the WSL2 ext4 filesystem should give you both inotify support as well as near native file access speeds.

[Nasser]

according to vxlabs.com/2019/12/06/wsl2-io-measurements best performance is with WSL-2 running on ext4 (i.e. inside WSL-2) and not outside (i.e. NTFS). They say it is faster than WSL-1 on lxfs or NTFS.