gunicorn autoreload on source change

Question

Finally I migrated my development env from runserver to gunicorn/nginx.

It'd be convenient to replicate the autoreload feature of runserver to gunicorn, so the server automatically restarts when source changes. Otherwise I have to restart the server manually with kill -HUP.

Any way to avoid the manual restart?

Solution 1

While this is old question you need to know that ever since version 19.0 gunicorn has had the --reload option. So now no third party tools are needed.

Solution 2

One option would be to use the --max-requests to limit each spawned process to serving only one request by adding --max-requests 1 to the startup options. Every newly spawned process should see your code changes and in a development environment the extra startup time per request should be negligible.

Solution 3

Bryan Helmig came up with this and I modified it to use run_gunicorn instead of launching gunicorn directly, to make it possible to just cut and paste these 3 commands into a shell in your django project root folder (with your virtualenv activated):

pip install watchdog -U
watchmedo shell-command --patterns="*.py;*.html;*.css;*.js" --recursive --command='echo "${watch_src_path}" && kill -HUP `cat gunicorn.pid`' . &
python manage.py run_gunicorn 127.0.0.1:80 --pid=gunicorn.pid

Solution 4

I use git push to deploy to production and set up git hooks to run a script. The advantage of this approach is you can also do your migration and package installation at the same time. https://mikeeverhart.net/2013/01/using-git-to-deploy-code/

mkdir -p /home/git/project_name.git
cd /home/git/project_name.git
git init --bare

Then create a script /home/git/project_name.git/hooks/post-receive.

#!/bin/bash
GIT_WORK_TREE=/path/to/project git checkout -f
source /path/to/virtualenv/activate
pip install -r /path/to/project/requirements.txt
python /path/to/project/manage.py migrate
sudo supervisorctl restart project_name

Make sure to chmod u+x post-receive, and add user to sudoers. Allow it to run sudo supervisorctl without password. https://www.cyberciti.biz/faq/linux-unix-running-sudo-command-without-a-password/

From my local / development server, I set up git remote that allows me to push to the production server

git remote add production ssh://user_name@production-server/home/git/project_name.git

# initial push
git push production +master:refs/heads/master

# subsequent push
git push production master

As a bonus, you will get to see all the prompts as the script is running. So you will see if there is any issue with the migration/package installation/supervisor restart.

View more solutions

Comments

[Paolo]

Errata: in my env gunicorn is managed/monitored by supervisord, so I wouldn't really kill -HUP the process PID, but use supervisorctl instead. Don't think this changes a lot, though.

[K Z]

github.com/benoitc/gunicorn/issues/154 has some solutions

[hobs]

Just used it for myself on fedora 15 with Django 1.5.4, gunicorn 18.0, watchdog 0.6, bash 4.2.

[hobs]

Don't forget to put your IP or FQDN and port in place of 127.0.0.1:80, if needed.

[hobs]

@Guandalino, any luck? It's been working well for me for a couple weeks now. Only time I need to manually restart is when I change settings.py, models.py (migration required), or the source code of some external app not in my watchmedo patterns.

[hobs]

Nice, elegant trick for dev env. Can't be used on prod... but you may not want autoreload on prod anyway, unless you do "continuous deployment". If you do, Bryan Helmig's approach is better even though it requires the pipable package, watchdog.

[Paolo]

Thanks for the reminder. But I don't want to cast my vote on other's success. Why this (unnecessary) hurry? Am I violating some StackOverflow rule? If so please let me know how to remediate.

[hobs]

No worries. Definitely not violating an SO rule, it's just considerate/solicitous/thoughtful to put effort/priority into evaluating helpful answers. It looks like Dave and I took our sweet time helping you out (many months), so my sense of urgency on getting you to verify our solutions is disproportionate -- I'm overly eager to know if there are hidden flaws in the way I've configured my server and if I should switch to Dave's approach. Happy Holidays!

[Blaise]

It will take ~ 3 seconds to boot a new worker, which is too slow for me. (mid-2009 MBP)

[J-bob]

agreed. The other answers may work, but this is by far the simplest and it's not a workaround. It's exactly what the OP wanted.

[sofly]

I don't believe there is a --reload option built into gunicorn. Where did you find this? Their docs say to reload the config, send a HUP (killall -HUP procname will work fine) to have new workers started and old ones gracefully shut down.

[sofly]

Thanks @Guandalino, I must have missed it. Interesting to note though, that they say "This setting is intended for development." This would obviously work for production in some cases, but could also be problematic for a lot of others. Yes I did see below that you are seemingly uninterested in production /deploy.

[juan Isaza]

How to do it in an easy way for production servers?

[Dmitry Ziolkovskiy]

@juanIsaza you should never use such functionality on production. If you think you need it - you need to rethink your approach for dev or deployment.

[Mykhailo Seniutovych]

--reload will not work when you ship byte code .pyc files for your web application, and then try to update these byte code files

[trallnag]

Akkschually it will help to have inotify installed to increase performance a bit. See docs.gunicorn.org/en/stable/settings.html#reload

[curtisp]

note to use shebang #!/bin/bash as noted above instead of #!/bin/sh which is what Git post-receive example had!