uwsgi: What defines the number of workers/process that a django app needs?

Question

I have a question for sysadmins and developers. I see that uWSGI allows me to set the number or workers or processes when running uWSGI and I had read that it depends on the installed machine, so I have the next questions:

1. What are the rules that define the number of workers for the machine?
2. When used with nginx, does the config worker_processes in nginx.conf affects this?
3. When used with Celery and Redis, is the concurrency related to this?
4. What about the thread safety in this setup? (I have seen cases in my app where 1 request executes 1 task and the result is 2 calls to celery with this task.)

Solution

• What are the rules that define the number of workers for the machine?

  From the uWsgi docs:

  There is no magic rule for setting the number of processes or threads. It is application and system dependent. Do not think using simple math like 2*cpucores will be enough. You need to experiment with various setup an constantly monitor your app. uwsgitop could be a great tool to find the best value.

• When using with nginx, the config "worker_processes" in nginx.conf afects this?

  That setting is only related to nginx and doesn't directly affect uwsgi. However, you will probably need to tinker with this setting because you don't want to have twenty nginx worker_processes and only one uwsgi worker.

• When using with celery+redis the "concurrency" is related with this?

  I usually spin up the Celery daemon with django's manage.py, so this will operate outside of uwsgi.

• What about the thread safety in this setup?

  I'd have to know more details/specifics to comment.