WSGI says "permissions denied" on my Ubuntu server, no WSGISocketPrefix setting works

Question

I am trying to run Apache2 with mod_wsgi supporting daemon processes on Ubuntu 10.04.3 LTS (lucid).

The problem is, I am not able to find out a working configuration for WSGISocketPrefix directive. My settings are:

<VirtualHost *:80>
    ...

    WSGIDaemonProcess myapp threads=5
    WSGIScriptAlias / /var/www/myapp/myapp.wsgi
    <Directory /var/www/myapp>
        WSGIProcessGroup myapp
        WSGIApplicationGroup %{GLOBAL}
        WSGIScriptReloading On
        Order deny,allow
        Allow from all
    </Directory>
</VirtualHost>

Apache runs as root. I am using Flask Python framework, so I followed this tutorial: http://flask.pocoo.org/docs/deploying/mod_wsgi/#configuring-apache. Without any other settings I got 503 Service Temporarily Unavailable HTTP error. In Apache error log I got this message:

[Mon Oct 17 15:24:24 2011] [error] [client 90.181.85.69] (13)Permission denied: mod_wsgi (pid=21805): Unable to connect to WSGI daemon process 'kvinono' on '/var/run/apache2/wsgi.16282.4.1.sock' after multiple attempts.

Then I found this: https://code.google.com/p/modwsgi/wiki/ConfigurationIssues#Location_Of_UNIX_Sockets, so I tried to set WSGISocketPrefix to any value that came in my mind and always tried to restart/reload Apache and looked if it works. Never worked, always permission error, only at different location. I tried to set user/group to WSGI process:

WSGIDaemonProcess myapp user=www-data group=www-data threads=5

...and at the same time to set the right permissions on folders like /var/run/wsgi and similar, but it didn't help. Running WSGI daemon process as root/root wasn't possible, Apache won't let me do that. Actually, WSGI was able to write and it really wrote the socket file into folder when all permissions were set well, but it didn't solved the error. Even with existing socket file it yileded completely the same permission denied error. After several more attempts and combinations I tried to set WSGISocketPrefix even to /tmp. Again, WSGI was able to create the socket file, but still "crashed" on the error above.

I am totally desperate now :-( If you suggest me to dance around a fire and sing some shaman curses, I am ready to do that only if it helps to solve the problem.

Solution 1

You are using ITK MPM for Apache. You will need to be using mod_wsgi 3.3 or later, which contains fix:

When compiled against ITK MPM for Apache, if using daemon mode, the listener socket for daemon process will be marked as being owned by the same user that daemon process runs. This will at least allow a request handled under ITK MPM to be directed to daemon process owned by same user as script. See issue:

http://code.google.com/p/modwsgi/issues/detail?id=187

You cannot though just use binary supplied by operating system as those available are likely only going to be for worker and prefork MPM. For ITK MPM you will need to compile mod_wsgi from source code and you MUST have the appropriate headers files for the ITK MPM installed and not those for worker or prefork MPM. This is because mod_wsgi source code has:

    if (!geteuid()) {
#if defined(MPM_ITK)
        if (chown(process->socket, process->uid, -1) < 0) {
#else
        if (chown(process->socket, ap_unixd_config.user_id, -1) < 0) {
#endif
            ap_log_error(APLOG_MARK, APLOG_ALERT, errno, wsgi_server,
                         "mod_wsgi (pid=%d): Couldn't change owner of unix "
                         "domain socket '%s'.", getpid(),
                         process->socket);
            return -1;
        }
    }

IOW, it is a compile time choice as to how to setup permissions with it only being set for ITK MPM correctly if ITK MPM headers are installed properly and so MPM_ITK #define found.

In summary, you will need to do the following:

(1) Ensure that ITK MPM header files installed. If using binary package for Apache, see if there is an ITK variant of the Apache dev package.

(2) Compile and install mod_wsgi from source code available in publically downloadable mod_wsgi 3.3 source package

The mod_wsgi source code package and installation instructions are available from:

http://www.modwsgi.org

Solution 2

Graham Dumpleton's answer might work, I haven't tried it. I'm using mod_wsgi 3.2 and have the same issue.

The problem seems to be the WSGISocketPrefix starts in /etc/httpd (or /etc/apache2) and uses simple concatenation. So when you add to your config

WSGISocketPrefix /var/run/wsgi

it ends up trying to put the file in /etc/httpd//var/run/wsgi. For some reason it doesn't log this error.

The workaround? I got it working like so:

WSGISocketPrefix ../../var/run/wsgi

I discovered this by using "var/run/wsgi" (without leading /) as my prefix and saw in the logs:

[Thu Feb 14 14:50:28 2013] [alert] (2)No such file or directory: mod_wsgi (pid=18702): Couldn't bind unix domain socket '/etc/httpd/var/run/wsgi.18702.0.1.sock'.

Well hopefully this is fixed in the latest version but at least we can stick with our OS distributed package.

Comments

[Graham Dumpleton]

What Apache MPM are you using? Are you using something other than worker or prefork MPM? If you are using ITK MPM you need to use version out of mod_wsgi source code repository to avoid the problem you are seeing.

[Honza Javorek]

I'm not such a specialist about Apache, so I'm not sure what MPM exactly is, but I've found this: httpd.apache.org/docs/2.0/mpm.html and it says I can list compiled modules and it should contain also MPM. My Apache returns this: core.c, mod_log_config.c, mod_logio.c, itk.c, http_core.c, mod_so.c. So, I assume my Apache is that ITK kind of thing...! However, I do not completely understand the sentence "you need to use version out of mod_wsgi source code repository". Could you expand it to a more explanatory answer? Seems you know the solution of my problem.

[Graham Dumpleton]

Actually, doesn't need version from source repository, but you will need to have mod_wsgi 3.3 and will need to be compiled against Apache with ITK. You can't just use binary package from OS distribution built for worker or prefork MPM. See separate answer.

[Honza Javorek]

I just tried what you suggested with WSGISocketPrefix set to /tmp/wsgi to avoid any real permission collisions. I added slashes to the end of WSGI paths and directories. Nothing changed, still getting the same permission error.

[Graham Dumpleton]

Never add a trailing slash after the the WSGI script file given as last argument to WSGIScriptAlias. It is wrong and the documentation at code.google.com/p/modwsgi/wiki/… says as much.

[Honza Javorek]

Seems to be a solution, but I am not able to find a way how to fullfill (1) at my Ubuntu server. It has packages for ITK version of Apache and it has dev package for Apache, but it probably doesn't match. I need specifically headers for ITK Apache. I have no idea how and where to get them.

[Honza Javorek]

If anyone else is trying to fix this problem (as me), following article can be useful: lonesomedev.com/?p=169 Describes most of the work of installing and compiling own mod_wsgi (although not with ITK).