Getting Node-gyp working on windows 10

Question

I'm just learning nodejs on windows. It seems that the node-gyp package is incredibly painful to set up. I've tried many guides online but I feel like I am playing a guessing game. The windows version is quite fresh , only a week or so old.

The official page ( https://github.com/nodejs/node-gyp ) says:

( trying to take the least complicated path )

npm install -g node-gyp

npm install --global --production windows-build-tools

If the above didn't work" go to https://github.com/Microsoft/nodejs-guidelines/blob/master/windows-environment.md#compiling-native-addon-modules

---

If I just run node-gyp, I get

Cannot find module:

'C:\Program Files\nodejs\node_modules\npm\node_modules\node_modules\node-gyp\bin\node-gyp.js' ( etc ). Even though that file exists.

I've even tried uninstalling node , clearing out %appdata% cache etc and removing other things.

---

If I try to install somethin dependent on node-gyp , I get :

..\src\ursaNative.cc(157): warning C4244: ( etc ) ..

ERR! stack Error: msbuild failed with exit code: 1 gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:258:23)

---

I have spent many , many hours on this - I cannot believe how painful this is.

I have

• Installed the VS Tools ( as above )
• Installed the C++ Compiler
• installed Build tools https://www.visualstudio.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15
• Tried setting the environment to VS2015 ( and VS 2017 )
• tried to follow many stack overflow fixes and posts. Surely there is a sure path to get this going.
• Various combinations of uninstalling / re-installing locally and globally.

Any ideas where to go from here?

Solution 1

Got to this post trying to get a correct set-up for node-gyp, after upgrading Node.js from v5.X to v12.X by overriding previous installation, and upgrading node-gyp.

It seems fairly easy to miss some step when you already have a Visual Studio environment and one or more versions of Python installed.

I will try to synthesise here the steps that worked for me.

CONTEXT

Below the details as per configuration:

+-------+-----------------------------+-------------+-------------------------------------+
|       | Component                   | Name        | Version                             |
+-------+-----------------------------+-------------+-------------------------------------+
| (1)   | Platform                    | Windows 10  | NT 10.0.18362                       |
| (2)   | Target run-time environment | Node.js     | v12.16.1                            |
| (3)   | Packet Manager              | npm         | v6.13.4                             |
| (4)   | CLI Toolchain for compiling | node-gyp    | v6.0.1                              |
| (4.a) | Compiler Language           | Python      | v2.7.0                              |
| (4.b) | Project Builder             | MSBuild.exe | Microsoft (R) Build Engine for .NET |
|       |                             |             | version 16.2.32702+c4012a063        |
|       |                             |             | 16.200.19.32702                     |
+-------+-----------------------------+-------------+-------------------------------------+

Guideline

You might have already walked through some of the steps outlined here. If you incurred in some issues, you may be able to identify at which stage something went wrong by reviewing the steps in detail:

References

1. node-gyp on GitHub: README.md (Generate Your Projects)
2. A Comprehensive Guide to Fixing Node-Gyp Issues on Windows (by Joe Bustamante; March 27, 2019)
3. Setting up None.js on Windows 10 (by Ferenc Hámori; May 17, 2016)
4. Using Python on Windows (official documentation)

(1) Node.js

This guideline assumes that git is already installed in your machine (you will find the installer for Windows here anyways).

• Install Node.js: download the last LTS version for your platform (Windows Installer .msi should work)

To verify it is correctly installed, in a test folder, add a file test.js with this javascript code: console.log("Node is installed!");, and in a terminal run: node test.js. You should be prompted with Node is installed!.

(2) npm

• upgrade npm to the latest stable version using the npm-windows-upgrade package

Run as Administrator a PowerShell terminal:

PS C:\WINDOWS\system32> npm-windows-upgrade

You will be prompted to choose a version. The greatest available version among the options should be fine.

If you miss this package you might want to follow the instructions to install it by following the steps outlined in the GitHub repository, which basically are to type the following in a PowerShell console running as Administrator:

PS C:\WINDOWS\system32> Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force
PS C:\WINDOWS\system32> npm install --global --production npm-windows-upgrade
PS C:\WINDOWS\system32> npm-windows-upgrade

(3) Visual Studio

If you have Visual Studio already installed, you might skip this step, unless you wanted to also upgrade to the newest version.

In some posts, you might have read that you can achieve this step by simply using the following command line, after installing node-gyp:

npm install --global --production windows-build-tools --vs2015

However, you could opt to achieve this by other means:

1. you may have already an installation of Visual Studio and just want to configure node-gyp to use it
2. you may prefer to do a separated installation of Visual Studio and later on configure node-gyp to use it

This is really up to you. At a later stage, in this guideline, we will walk through the steps to configure the node-gyp to use a specific Visual Studio version.

(4) Python

In the installation guideline for node-gyp (GitHub official repo) specifies which versions of Python are currently compatible with the latest node-gyp version on Unix and macOS. However it does not explain for Windows platforms (as at 1st of March 2020).

Although lack of documentation on this point, by having a look at other users' issues with this, it is fair to assume that on Windows platforms, node-gyp is only supported for Python v2.7.X (reference).

• download Python 2.7.X here: you can choose the bugfix release noted with an available link at the beginning.
• do a normal installation, and take note of the folder (preferably in a common folder where you will have all the python versions)

(5) node-gyp

Now it is the moment to correctly set up your node-gyp configuration.

If you haven not installed it as yet:

npm install --global node-gyp

(5.1) Set the Python version

According the documentation:

If the NODE_GYP_FORCE_PYTHON environment variable is set to the path of a Python executable, it will be used instead of any of the other configured or builtin Python search paths. If it's not a compatible version, no further searching will be done.

1. identify the full path to the C:\full_path\Python2.7.X\python.exe file (by full path we mean all: the folder path + the target file python.exe)
2. go to Control Panel -> System and open Advanced System Settings, tab Advanced
3. at the bottom, click the button Environmental Variables...
4. a new panel pops up with two big sections: User Variables (only for the process owner), and System Variables (applicable to all the processes)
5. on the System Variables, create a New entry
6. name it NODE_GYP_FORCE_PYTHON, and as value use the full path to the python.exe file version 2.7.X, and click OK and you are done

• for the variable to be available in the environment of your PowerShell terminal, you will need to close it and reopen a new terminal

7. you have just fixed the python version to be used for node-gyp in your system

Alternatively you can use the command line below:

npm config set python /path/to/executable/python

(5.2) Set the Visual Studio Build Tools version

This step should be fairly easy:

1. identify the year of your Visual Studio edition (i.e. 2015, 2017, 2019)
2. use it in the year part of the command line below:

npm config set msvs_version year

For example, if you want it to use the MSBuild of 2019, use the command below:

npm config set msvs_version 2019

That must have done it.

(6) Testing node-gyp by creating a simple add-on in C++

References

1. Mastering Node.js: Build robust and scalable real-time server-side web (by Sandro Pasquali & Kevin Faaborg, 2017, Packt Publishing): adapted example from here
2. C++ Addons (official documentation)
3. V8 Embed (explanation on how V8 is embedded in C++)
4. V8::FunctinonCallbackInfo Class Template reference for Node.js v12.0

Hands on work

In a test folder, create test\hello_module subfolder with the following empty files:

• hello_module\hello.cc (our source C++ native code)
• hello_module\binding.gyp (instruction file for node-gyp)
• hello_module\index.js (the wrapper)

In a terminal initialise the package by npm ini. You can choose the offered default value by just pressing Enter in all of them:

test\hello_module> npm ini 

Now fill the files in with the contents specified below. Please, leave the index.js for the end, since we will be compiling before using it.

---

The hello.cc file content:

#include <node.h>

namespace hello_module {
  using v8::FunctionCallbackInfo;
  using v8::Isolate;
  using v8::Local;
  using v8::Object;
  using v8::String;
  using v8::Value;
  
  void sayHello(const FunctionCallbackInfo<Value>& args) {
    Isolate* isolate = args.GetIsolate();
    args.GetReturnValue().Set(String::NewFromUtf8(isolate, "Hello World!"));
  }
  
  // the initialization function for hello_module
  void init(Local<Object> exports) {
    NODE_SET_METHOD(exports, "sayHello", sayHello);
  }
  
  // node.h C++ macro to export the initialization function
  // (macros should not end by semicolon)
  NODE_MODULE(NODE_GYP_MODULE_NAME, init)
}

The binding.gyp file content:

{
  "targets": [
    {
      "target_name": "hello",
      "sources": [ "hello.cc" ]
    }
  ]
}

---

You can leave the index.js file for the end.

Now, let's build the project:

test\hello_module> node-gyp configure
 gyp info it worked if it ends with ok
 gyp info using [email protected]
 gyp info using [email protected] | win32 | x64
 gyp info find Python using Python version 2.7.0 found at "C:\python\2.7.0\python.exe"
 gyp info find VS using VS2019 (16.2.29123.88) found at:
 gyp info find VS "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community"
 # .. omited .. #
 gyp info ok

The test\hello_module\build folder should have been created with the basic project files to compile a C++ solution (this is what node-gyp basically aims to: that you can use any C++ compiler without having to use the GUI; in this case Visual Studio).

Now, let's build the addon:

test\hello_module> node-gyp build

At the end of both commands you should read gyp info ok to know that everything was okay (you might not see it on a PowerShell terminal because of the blue background; if so, you can edit the Properties of the window and change Screen Background to black).

This command should have created the test\hello_module\build\Release folder with the hello.node file.

Notes:

1. If everything went ok, you are done in verifying your node-gyp installation: it works!

• once you got this simple add-on to work, if you are still having issues with some package(s), that might be related to that package alone, or some of its dependencies, but your node or node-gyp configuration are okay

2. if you encountered problems during the build, you might use the documentation to troubleshoot the source of the problem, or open a new post specifying which the errors you encountered are, and someone might be able to help

(7) Wrap and use the add-on in C++

This is an extra step. As you might have got here, why leaving it like this?

Now let's write the wrapper hello_module\index.js file:

const helloAddon = require('./build/Release/hello.node');
module.exports = helloAddon;

And in the test folder, create the test\hello_world.js file that uses our addon:

const {sayHello} = require('./hello_module');
console.log(sayHello())

And in the terminal:

test> node hello_world.js

You should see Hello World! prompted on the screen.

Hope this helps anyone having issues to identify where exactly the configuration of node-gyp failed to meet requirements.

Solution 2

your node-gyp module is initialised from the NPM but there is a fair chance that its correct path is not added to the environment variables in your system, Just add it to the system variables. Check the path variable if it has "C://Users/abc/AppData/Roaming/npm" if not, Please add it.

Also check if python is added to your environment variable.

Cheers!

Solution 3

This was raised in issue 1463. It seems to be a bug from following the docs instructions for node-gyp install -g and npm install --global --production windows-build-tools

I found a very simple solution.

If node_gyp is on your disk as it should be, like so: "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js"

But an error message expects this: "C:\Program Files\nodejs\node_modules\npm\node_modules\node_modules\node-gyp\bin\node-gyp.js"

You simply add an extra node_modules folder beneath "C:\Program Files\nodejs\node_modules\npm\node_modules\"

Then you copy the entire existing \node-gyp folder into the new \node_modules\node_modules folder.

You have a redundant node-gyp folder, but the configure and build steps will now work as expected.

At least that worked for me.