Python3 threading with uWSGI

Question

I wasted many time but couldn't find a solution.
If i use threads in my app deployed with uwsgi, they aren't sync.

Here simple code for an example(wsgi.py):

from time import sleep
import threading

i = 0
def daemon():
  global i
  while True:
    i += 1
    print(i)
    sleep(3)
th = threading.Thread(target=daemon, args=())
th.start()

def application(environ, start_response):
  start_response('200 OK', [('Content-Type','text/html')])
  return [str(i).encode()]

And when I run this app the i increases in log, but I always get 1 when a make request from browser.(Or get 0 if I move sleep(3) before i first increment)
I tried uwsgi.thread decorator, but got the same result.

uwsgi config:

[uwsgi]
socket = 127.0.0.1:3034
plugins-dir = /srv/uwsgi
plugin = python34
uid = py3utils
gid = py3utils
chdir = /srv/python/3/py3utils/tht/app/
wsgi-file = wsgi.py
enable-threads = true
daemonize = %(chdir)/../uwsgi.log
master = true
die-on-term = true
touch-reload = ../uwsgi_restart.txt

*sorry for my English

Solution 1

This happens because after importing your application the master process forks into a worker:

spawned uWSGI master process (pid: 7167)
spawned uWSGI worker 1 (pid: 7169, cores: 1)
spawned uWSGI http 1 (pid: 7170)

So your thread which prints i is running in master process, and your requests are processed by the worker. The worker during the fork sees i equal to 1. If you move sleep before incrementing i the process manages to fork before the first increment.

Threads except the main one are not copied during a fork, so i does not increment in the worker.

You should use something like uwsgidecorators.thread:

from time import sleep
import threading
import uwsgidecorators

i = 0

@uwsgidecorators.postfork
@uwsgidecorators.thread
def daemon():
  global i
  while True:
    i += 1
    print(i)
    sleep(3)

def application(environ, start_response):
  start_response('200 OK', [('Content-Type','text/html')])
  return [str(i).encode()]

Or use:

[uwsgi]
master = false

Solution 2

Python threading is disabled by default in uwsgi, you can enable it by adding option --enable-threads:

uwsgi --http :8090 --wsgi-file uwsgi_test.py --enable-threads

It works in my test environment.

Comments

[n4nn31355]

You saved my sanity. master = false enough for my current task. But a quick question: if I use your code with @postfork I got print result in log twice for each iteration. Why?