Source shell script into makefile

Question

I have a bash shell script which I usually source into my shell, with lots of environment variables defined, which are not exported. I do not want to:

1. Export the variables, because this would make the exportable environment too big, and eventually make the whole system slower (it must be exported when running every command from the shell)
2. Redefine those variables in the makefile (DRY)

I would like to source the same shell script into the environment of the makefile, so that I can access those variables. Is this possible? How can I do that? Ideally I would do in the makefile:

source setup-env.sh

There is not source command for makefiles, but maybe something equivalent? Any special hack I can use to simulate the same effect?

Solution

As per the additional question in the comment, here is one way to effectively mark the whole environment as exported:

for var in $(compgen -v); do export $var; done

compgen -v simply outputs all variable names, as per the bash manual, section 8.7 Programmable Completion Builtins. Then we simply loop over this list and export each one.

Credit to https://stackoverflow.com/a/16337687/2113226 - compgen is new to me.

---

There are two ways I can think of to integrate this into your make workflow:

- Shell script wrapper

Simply write a shell script which sources your setup-env.sh, exports all variables as above, then calls make itself. Something like:

#!/bin/bash

./source setup-env.sh
for var in $(compgen -v); do export $var; done
make $@

- Recursive make

It may be that you don't want a shell script wrapper, and want to directly invoke make for whatever reason. You can do this all in one Makefile which calls itself recursively:

$(info MAKELEVEL=$(MAKELEVEL) myvar=$(myvar))
ifeq ($(MAKELEVEL), 0)
all:
    bash -c "source ./setup-env.sh; \
    for var in \$$(compgen -v); do export \$$var; done; \
    $(MAKE) $@"
else
all: myprog

myprog:
    echo "Recipe for myprog.  myvar=$(myvar)"
endif

Output for this Makefile is:

$ make
MAKELEVEL=0 myvar=
bash -c "source ./setup-env.sh; \
    for var in \$(compgen -v); do export \$var; done; \
    make all"
MAKELEVEL=1 myvar=Hello World
make[1]: Entering directory `/home/ubuntu/makesource'
echo "Recipe for myprog.  myvar=Hello World"
Recipe for myprog.  myvar=Hello World
make[1]: Leaving directory `/home/ubuntu/makesource'
$ 

We check the GNU Make builtin variable MAKELEVEL to see what level of recursion we are at. if the level is 0, then we recursively call make for all targets, but first source ./setup-env.sh and export all variables. If the recursion level is anything else, we just do the normal makefile stuff, but you see that the variables you need are now available. This is highlighted by the $(info ) line at the top of the Makefile, which shows the recursion level, and the value (or not) of myvar.

Notes:

• We have to use bash -c because compgen is strictly a bash builtin, and not available in Posix mode - i.e. when make invokes the shell as sh -c by default.

• The $ in the first all: recipe need to be escaped very carefully. The $$ escapes the $ from being expanded by make, and the \$$ escapes the $ from being expanded by the implicit sh

• There is plenty of literature arguing that "Recursive make is considered harmful". E.g. http://aegis.sourceforge.net/auug97.pdf