Supervisor not working with Gunicorn + Flask

Question

I am trying to run Gunicorn from Supervisor in an Ubuntu 12.04 system. Gunicorn runs a Flask app (simple REST web service tested with Flask's embedded server). I have installed Gunicorn by clonning GIT repo, trying to avoid 'apt-get install' because it runs Gunicorn server when installs it. I do not want it running, it will be run by Supervisor only.

So after install it, if I try:

cd /usr/local/bin
gunicorn my_app:app -c /path/to/gu_config_file

Gunicorn works. Then I kill it. Note config file without extension, because with '.py' extension does not work for me. So I edit Supervisor's config file like:

[program:gunicorn]
command=/usr/local/bin/gunicorn my_app:app -c /path/to/.gu_setup
directory=/usr/local/bin/
autostart=true
autorestart=true
redirect_stderr=True

And update changes in Supervisor:

supervisorctl reread
# gunicorn: changed
supervisorctl update
# gunicorn: stopped
# gunicorn: updated process group

Detects changes in file and works for Gunicorn program. Ok, but then I try to start it:

supervisorctl start gunicorn

Getting an annoying:

gunicorn: ERROR (abnormal termination)

Checking supervisor's log:

2013-03-08 13:07:22,378 INFO spawned: 'gunicorn' with pid 3355
2013-03-08 13:07:22,916 INFO exited: gunicorn (exit status 3; not expected)
2013-03-08 13:07:23,918 INFO spawned: 'gunicorn' with pid 3361
2013-03-08 13:07:24,492 INFO exited: gunicorn (exit status 3; not expected)
2013-03-08 13:07:26,496 INFO spawned: 'gunicorn' with pid 3367
2013-03-08 13:07:27,078 INFO exited: gunicorn (exit status 3; not expected)
2013-03-08 13:07:30,085 INFO spawned: 'gunicorn' with pid 3373
2013-03-08 13:07:30,628 INFO exited: gunicorn (exit status 3; not expected)
2013-03-08 13:07:31,630 INFO gave up: gunicorn entered FATAL state, too many start retries too quickly

I do not know what to do right now... Can you help me? Thx a lot!

EDIT: sorry I forgot to say I have exported PYTHONPATH variable as:

export PYTHONPATH=/usr/local/bin:/usr/local/lib/project

'my_app' is in /usr/local/bin. The lib path is needed for other modules. I have edited also Supervisor config file to indicate environmental variable, like:

environment=PYTHONPATH=/usr/local/bin:/usr/local/lib/project/

But did not work.

EDIT 2: as @robertklep suggest in his comment, this is log's output:

Traceback (most recent call last):
  File "/tmp/gunicorn/gunicorn/arbiter.py", line 485, in spawn_worker
    worker.init_process()
  File "/tmp/gunicorn/gunicorn/workers/base.py", line 100, in init_process
    self.wsgi = self.app.wsgi()
  File "/tmp/gunicorn/gunicorn/app/base.py", line 103, in wsgi
    self.callable = self.load()
  File "/tmp/gunicorn/gunicorn/app/wsgiapp.py", line 25, in load
    return util.import_app(self.app_uri)
  File "/tmp/gunicorn/gunicorn/util.py", line 369, in import_app
    __import__(module)
  File "/usr/local/bin/my_app.py", line 4, in <module>
    import const
ImportError: No module named const
2013-03-08 13:29:35 [3670] [INFO] Worker exiting (pid: 3670)
2013-03-08 13:29:36 [3665] [INFO] Shutting down: Master
2013-03-08 13:29:36 [3665] [INFO] Reason: Worker failed to boot.

'const' module is in /usr/local/lib/project...

Solution 1

I don't see you setting the environment in your supervisor config file:

[program:gunicorn]
environment=PYTHONPATH=/usr/local/bin:/usr/local/lib/project
command=/usr/local/bin/gunicorn my_app:app -c /path/to/.gu_setup
...

If that doesn't work, try starting gunicorn in debug mode:

command=/usr/local/bin/gunicorn --debug --log-level debug my_app:app -c /path/to/.gu_setup

Or pass the path directly to gunicorn:

command=/usr/local/bin/gunicorn --pythonpath /usr/local/bin,/usr/local/lib/project my_app:app -c /path/to/.gu_setup

EDIT: gunicorn's --pythonpath is broken, you can only pass one directory:

command=/usr/local/bin/gunicorn --pythonpath /usr/local/lib/project my_app:app -c /path/to/.gu_setup

Solution 2

It's not necessary pass --pythonpath. If you work virtuanenv you put add where is gunicorn. Example:

command=/home/virtualenv/bin/gunicorn application:app -c /home/virtualenv/deploy/gunicorn.conf.py

And directory is when Flask code is, Example:

directory=/home/virtualenv/myapp

Remember user is root!

user=root

Comments

[Alberto Megía]

Yes, I did that and were editing my question at the same time you response me... sorry! I am doing what you said, wait for it! :)

[Alberto Megía]

So is telling me it cannot find 'const' module, but it is in /usr/local/lib/project, added to environment in Supervisor's config file...

[Alberto Megía]

The same thing: no module named 'const'. But it is there, beeing also compiled by python! (pyc file appeared). I think it is a problem with Supervisor cd, because I can run it with Gunicorn only setting PYTHONPATH by myself...

[Alberto Megía]

Reading Supervisor doc: "Subprocesses will inherit the environment of the shell used to start the supervisord program", "No shell is executed by supervisord when it runs a subprocess, so environment variables such as USER, PATH, HOME, SHELL, LOGNAME, etc. are not changed from their defaults or otherwise reassigned." and "If you need to set environment variables for a particular program that might otherwise be set by a shell invocation for a particular user, you must do it explicitly within the environment= program config option." (supervisord.org/subprocess.html#subprocess-environment)

[robertklep]

I'm just noticing that gunicorn's --pythonpath might actually be broken. Try setting just --pythonpath /usr/local/lib/project.

[Alberto Megía]

I think that worked, but I need to check it in a clean machine. I will tell it to you so you can edit your response and I will mark it! Thank you very much @robertklep

[Greg Wang]

Setting --pythonpath /path/to/project solved my problem. Thanks!