module load command in linux bash script

Question

I need to execute Python script in HPC cluster. Unfortunately, the default python version is just 2.6.6 and there is no numpy and scipy.

I can load these modules in command line

#module load /home/hw1u16/modules/2.7.3

and

module load /home/hw1u16/modules/1.6.2

However, when I write the bash script like this

module load /home/hw1u16/modules/2.7.3
module load /home/hw1u16/modules/1.6.2
python /home/hw1u16/project/trainAgent.py

It warns me

ModuleCmd_Load.c(200):ERROR:105: Unable to locate a modulefile for '/home/hw1u16/modules' ModuleCmd_Load.c(200):ERROR:105: Unable to locate a modulefile for '/home/hw1u16/modules' I don't know what's wrong, could any guys help me?

Solution 1

I had a similar problem, and found two solutions:

1. instead of running your script with sh yourscript.sh or ./yourscript.sh, you could run it as . yourscript.sh This will source the module correctly and run the script

2. if you don't want to use . yourscript.sh, you can modify your shebang from #!/bin/sh to #!/bin/bash as noted in DavidC's answer and run your script as ./yourscript.sh Note that running it as sh yourscript.sh will not work

Solution 2

Okay, I think I know where is the problem. Try type module from the shell to see how module it is currently defined in your system. You will receive two options: either it is an alias or a function. This is because the module command is an alias or shell function.

Say your script is the following running.sh :

#!/bin/bash  
module load python/2.7.3
python /home/hw1u16/project/trainAgent.py

(It is a good practice to add the shebang)

To sort out this problem you have two options:

• Option 1:

source the scitpt. In other words, do: source running.sh. This is exactly the same as typing the module command directly into your interactive shell. However, by doing ./running.sh, you are running a new, non-interactive shell. These generally do not have the standard aliases and shell functions set up.

• Option 2:

Find the initialization script that defines the module command and source it from the script

Comments

[skwang]

I have tried the source command, seems module avail can be use in bash script, but when I write module load gcc/4.6.2 or module load python/2.7.3. It warns me 'ython(9):ERROR:105: Unable to locate a modulefile for 'python/2.7.3'. `

[DavidC.]

@skwang This is the same error you reported before. Please, after writing the script as I wrote in the answer, do source running.sh and then ./running.sh in the same terminal

[Joshua Perrett]

The shebang solution did not work for me (it also doesn't seem to be something DavidC said). The first solution worked great though. Didn't realise . was an alias for source.