How to use environment variables in package.json

Question

Because we don't want sensitive data in the project code, including the package.json file, using environment variables would be a logical choice in my opinion.

Example package.json:

  "dependencies": {
    "accounting": "~0.4.0",
    "async": "~1.4.2",
    "my-private-module":"git+https://${BB_USER}:${BB_PASS}@bitbucket.org/foo/bar.git"

Is this possible?

The question is not if this is wise or not good, just if it's possible.

Solution 1

In case you use .env file, let's use grep or eval to get a value environment variable from the .env file.

Updated start2 as @Paul suggested:

"scripts": {
    "start": "NODE_ENV=$(grep NODE_ENV .env | cut -d '=' -f2) some_script",
    "start2": "eval $(grep '^NODE_ENV' .env) && some_script"
}

Solution 2

I have similar but different requirement. For me, I want to use environment variables in the scripts.

Instead of using the environment variables directly in package.json, I do:

"some-script": "./scripts/some-script.sh",

And in some-script.sh:

#!/bin/sh

npm run some-other-script -- --prop=$SOME_ENV_VAR

Solution 3

No, it's not possible. You should access the repo using git+ssh, and store a private key in ~/.ssh.

Your line then looks like:

"my-private-module":"git+ssh://[email protected]/foo/bar.git"

Which doesn't contain anything sensitive.

Solution 4

Here's how I managed to work around package.json to achieve the same purpose. It uses a script that reads from a custom section of package.json for URL modules, interpolates environment variables in them, and installs them with npm install --no-save (the --no-save could be omitted, depending on the usecase).

As a bonus: it tries to read the env variable from .env.json, which can be gitignore'd, and very useful for development.

1. Create a script that will read from a custom section of package.json

env-dependencies.js

const execSync = require('child_process').execSync
const pkg = require('./package.json')

if (!pkg.envDependencies) {
  return process.exit(0)
}

let env = Object.assign({}, process.env)

if (typeof pkg.envDependencies.localJSON === 'string') {
  try {
    Object.assign(env, require(pkg.envDependencies.localJSON))
  } catch (err) {
    console.log(`Could not read or parse pkg.envDependencies.localJSON. Processing with env only.`)
  }
}

if (typeof pkg.envDependencies.urls === 'undefined') {
  console.log(`pkg.envDependencies.urls not found or empty. Passing.`)
  process.exit(0)
}

if (
  !Array.isArray(pkg.envDependencies.urls) ||
  !(pkg.envDependencies.urls.every(url => typeof url === 'string'))
) {
  throw new Error(`pkg.envDependencies.urls should have a signature of String[]`)
}

const parsed = pkg.envDependencies.urls
  .map(url => url.replace(/\${([0-9a-zA-Z_]*)}/g, (_, varName) => {
    if (typeof env[varName] === 'string') {
      return env[varName]
    } else {
      throw new Error(`Could not read env variable ${varName} in url ${url}`)
    }
  }))
  .join(' ')

try {
  execSync('npm install --no-save ' + parsed, { stdio: [0, 1, 2] })
  process.exit(0)
} catch (err) {
  throw new Error('Could not install pkg.envDependencies. Are you sure the remote URLs all have a package.json?')
}

2. Add a "postinstall": "node env-dependencies.js" to your package.json, that way it will be run on every npm install

3. Add your private git repos to package.json using the URLs you want (note: they all must have a package.json at root!):

"envDependencies": {
  "localJSON": "./.env.json",
  "urls": [
    "git+https://${GITHUB_PERSONAL_ACCESS_TOKEN}@github.com/user/repo#semver:^2.0.0"
  ]
},

(the semver specifier #semver:^2.0.0 can be omitted, but refers to a git tag, which can be very useful, as it makes your git server a fully-fledge package manager)

4. npm install

Solution 5

No it isn't possible as npm does not treat any string values as any kind of templates.

It may be better to just use git+ssh (if your provider supports it) with an ssh agent.

View more solutions

Comments

[Zlatko]

Additionally you can even use a different ssh key and ssh config for this purpose, not the usual id_rsa.

[Steve Bennett]

Yep, I'd recommend that. (I did mean that with "store a private key", but could be clearer.)

[kaasdude]

Thanks! Yeah, it's Heroku :-S. So should be a custom build pack I guess. Docker will be ultimate environment in the end I think. Need to do this! Regards!

[kaasdude]

This .npmrc is able to interpret environment variables though! But was not able to combine, not mend to use it for these purposes I think...

[NikolaDjokic]

This is for setting variables to be uses by npm scripts - similar to config section in package.json. These variables can't be read by package.json

[Robert Skarżycki]

Could you tell how do you access prop in some-other-script?

[techguy2000]

It's been a while. Can you follow the highest voted answer here to see if that works? stackoverflow.com/questions/5499472/…

[saurabh]

anybody using docker for deployment shouldn't use this method

[Ashmah]

@monothorn - what could be a better way to do that? Do you have any suggestion... I am still figuring out if we don't use ssh, what could be a better way.

[saurabh]

@Ashmah HTTPS is the way to go but again you will have to restrict access for the token you have generated and make sure the repo is private. Other than these, you will have to understand if there are any more security concerns.

[Marko Rochevski]

This should be the top comment because it perfectly solves this.

[writofmandamus]

I've been using this but when you start using this in multiple places it gets very verbose. Is there a library that can make this shorter?

[bmacnaughton]

this is awesome, thanks. it works in "scripts". unfortunately it doesn't work in "config" which would be really useful. any idea why?

[bmacnaughton]

nm - it's the shell, not npm

[Paul]

You can also shorten this to eval $(grep '^NODE_ENV' .env) && some_script

[Henry Woody]

Two issues with this. 1: The question was about dependencies, not scripts. Scripts run in the shell so using an environment variable in a script is very different from using one in a regular package.json value. 2: (Maybe I'm missing something here, but) getting an environment variable's value via grep seems like a complicated way of just doing $NODE_ENV. Further, if NODE_ENV is already defined as an environment variable, then it doesn't need to be redefined. Seems like it just saves you from doing source .env (and you could use something like direnv to do it automatically for you).