Django + gunicorn + virtualenv + Supervisord issue

Question

I have a strange issue with my virtualenv + gunicorn setup, only when gunicorn is launched via supervisord. I do realize that it may very well be an issue with my supervisord and I would appreciate any feedback on a better place to ask for help...

In a nutshell : when I run gunicorn from my user shell, inside my virtualenv, everything is working flawlessly. I'm able to access all the views of my Django project.

When gunicorn is launched by supervisord at the system startup, everything is OK.

But, if I have to kill the gunicorn_django processes, or if I perform a supervisord restart, once that gunicorn_django has relaunched, every request is answered with a weird Traceback :

(...)
  File "/home/hc/prod/venv/lib/python2.6/site-packages/Django-1.2.5-py2.6.egg/django/db/__init__.py", line 77, in
    connection = connections[DEFAULT_DB_ALIAS]
  File "/home/hc/prod/venv/lib/python2.6/site-packages/Django-1.2.5-py2.6.egg/django/db/utils.py", line 92, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/home/hc/prod/venv/lib/python2.6/site-packages/Django-1.2.5-py2.6.egg/django/db/utils.py", line 50, in load_backend
    raise ImproperlyConfigured(error_msg)
TemplateSyntaxError: Caught ImproperlyConfigured while rendering: 'django.db.backends.postgresql_psycopg2' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
    'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: cannot import name utils

Full stack available here : http://pastebin.com/BJ5tNQ2N

I'm running...

• Ubuntu/maverick (up-to-date)
• Python = 2.6.6
• virtualenv = 1.5.1
• gunicorn = 0.12.0
• Django = 1.2.5
• psycopg2 = '2.4-beta2 (dt dec pq3 ext)'

gunicorn configuration :

backlog = 2048
bind = "127.0.0.1:8000"
pidfile = "/tmp/gunicorn-hc.pid"
daemon = True
debug = True
workers = 3
logfile = "/home/hc/prod/log/gunicorn.log"
loglevel = "info"

supervisord configuration :

[program:gunicorn]
directory=/home/hc/prod/hc
command=/home/hc/prod/venv/bin/gunicorn_django -c /home/hc/prod/hc/gunicorn.conf.py
user=hc
umask=022
autostart=True
autorestart=True
redirect_stderr=True

Any advice ? I've been stuck on this one for quite a while.

It seems like some weird memory limit, as I'm not enforcing anything special :

$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Solution

I tracked this one down with the help of davisp, a gunicorn developer. Thank you !

It was an environnement issue, caused by an invalid HOME setting in the environnement of the supervisor gunicorn subprocess.

I did not get anything on stderr until I put an "import psycopg2" in my settings.py file, which produced the following message on supervisor's stderr files for gunicorn.

Error: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg cache:

  [Errno 13] Permission denied: '/root/.python-eggs'"

I added the following line in my supervisor configuration file for gunicorn, and now Python finds its way to a u+writable egg cache. Everything is OK.

environment=HOME='/home/hc'