How to make an installed python module usable for another user?

Question

I'm running a bash script with php which will in turn run a python script but I get so many errors because it seems like www-data can't use the python packages I have installed for my own user. So how can I make a specific package available for www-data is it safe to do that? also is it safe to set www-data as the owner of a specific subtree of /var/www/html?

Here is the error I get when I run the following command:

sudo -u www-data ./bash_script_that_calls_runs_the_pythonscript.sh

Traceback (most recent call last):
  File "./file.py", line 5, in <module>
    from bs4 import BeautifulSoup
ImportError: No module named 'bs4'

But if I run:

./bash_script_that_calls_runs_the_pythonscript.sh

Everything will go fine.

And also:

~/.local/lib/python3.5/site-packages$ ll | grep bs4
drwxrwxr-x  5 me www-data  4096 Dec  2 15:37 bs4/
drwxrwxr-x  2 me www-data  4096 Dec  2 15:38 bs4-0.0.1.dist-info/

Please note that I've set the group recursively:

$ ll /home/me/.local/lib/python3.5/site-packages/bs4/
total 180
drwxrwxr-x  5 me www-data  4096 Dec  2 15:37 ./
drwx------ 51 me me        4096 Jan 16 04:33 ../
drwxrwxr-x  3 me www-data  4096 Dec  2 15:37 builder/
-rw-rw-r--  1 me www-data 29910 Dec  2 15:38 dammit.py
-rw-rw-r--  1 me www-data  6773 Dec  2 15:38 diagnose.py
-rw-rw-r--  1 me www-data 68798 Dec  2 15:38 element.py
-rw-rw-r--  1 me www-data 20394 Dec  2 15:38 __init__.py
drwxrwxr-x  2 me www-data  4096 Dec  2 15:37 __pycache__/
-rw-rw-r--  1 me www-data 30800 Dec  2 15:38 testing.py
drwxrwxr-x  3 me www-data  4096 Dec  2 15:37 tests/

The same goes for bs4-0.0.1.dist-info/

Solution

Answer:

Add -H or -i option to sudo:

sudo -i -u www-data ./bash_script_that_calls_runs_the_pythonscript.sh
sudo -H -u www-data ./bash_script_that_calls_runs_the_pythonscript.sh

Explain:

You can use python -m site to check importing paths. For example, in the outputs of sudo -u www-data python -m site, USER_SITE is not the expected directory bs4 installed.

sys.path = [
    '/',
    '/usr/local/lib/python3.6.2/lib/python36.zip',
    '/usr/local/lib/python3.6.2/lib/python3.6',
    '/usr/local/lib/python3.6.2/lib/python3.6/lib-dynload',
    '/usr/local/lib/python3.6.2/lib/python3.6/site-packages',
]
USER_BASE: '/root/.local' (doesn't exist)
USER_SITE: '/root/.local/lib/python3.6/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

With sudo -i or sudo -H, you can switch home directory to user www-data and find the correct USER_SITE. The outputs of sudo -i -u www-data python -m site may be:

sys.path = [
    '/home/www-data',
    '/usr/local/lib/python3.6.2/lib/python36.zip',
    '/usr/local/lib/python3.6.2/lib/python3.6',
    '/usr/local/lib/python3.6.2/lib/python3.6/lib-dynload',
    '/home/www-data/.local/lib/python3.6/site-packages',
    '/usr/local/lib/python3.6.2/lib/python3.6/site-packages',
]
USER_BASE: '/home/www-data/.local' (exists)
USER_SITE: '/home/www-data/.local/lib/python3.6/site-packages' (exists)
ENABLE_USER_SITE: True

Comments

[wjandrea]

Does www-data have execute permissions for the Python module?

[yukashima huksay]

@WJAndrea I think so(see the edit)

[wjandrea]

For a module in a directory, I think you need execute permissions for the __init__.py file. You might need execute permissions for all other .py files in the directory too.

[yukashima huksay]

@WJAndrea the permissions for the user and group seem to be the same. to I have to reload anything after changing the permissions to make changes take effect?

[yukashima huksay]

I don't want to run it from commandline! I want to run is from within a php script that is run by a request from the user.

[Ping Chu Hung]

When php received a request, can you print the outputs of python -m site in bash_script_that_calls_runs_the_pythonscript.sh?