Django, sleep() pauses all processes, but only if no GET parameter?

Question

Using Django (hosted by Webfaction), I have the following code

import time
def my_function(request):
    time.sleep(10)
    return HttpResponse("Done")

This is executed via Django when I go to my url, www.mysite.com

I enter the url twice, immediately after each other. The way I see it, both of these should finish after 10 seconds. However, the second call waits for the first one and finishes after 20 seconds.

If, however, I enter some dummy GET parameter, www.mysite.com?dummy=1 and www.mysite.com?dummy=2 then they both finish after 10 seconds. So it is possible for both of them to run simultaneously.

It's as though the scope of sleep() is somehow global?? Maybe entering a parameter makes them run as different processes instead of the same???

It is hosted by Webfaction. httpd.conf has:

KeepAlive Off
Listen 30961
MaxSpareThreads 3
MinSpareThreads 1
ServerLimit 1
SetEnvIf X-Forwarded-SSL on HTTPS=1
ThreadsPerChild 5

I do need to be able to use sleep() and trust that it isn't stopping everything. So, what's up and how to fix it?

Edit: Webfaction runs this using Apache.

Solution 1

As Gjordis pointed out, sleep will pause the current thread. I have looked at Webfaction and it looks like their are using WSGI for running the serving instance of Django. This means, every time a request comes in, Apache will look at how many worker processes (that are processes that each run a instance of Django) are currently running. If there are none/to view it will spawn additonally workers and hand the requests to them.

Here is what I think is happening in you situation:

• first GET request for resource A comes in. Apache uses a running worker (or starts a new one)
• the worker sleeps 10 seconds
• during this, a new request for resource A comes in. Apache sees it is requesting the same resource and sends it to the same worker as for request A. I guess the assumption here is that a worker that recently processes a request for a specific resource it is more likely that the worker has some information cached/preprocessed/whatever so it can handle this request faster
• this results in a 20 second block since there is only on worker that waits 2 times 10 seconds

This behavior makes completely sense 99% of the time so it's logically to do this on default.

However, if you change the requested resource for the second request (by adding GET parameter) Apache will assume that this is a different resource and will start another worker (since the first one is already "busy" (Apache can not know that you are not doing any hard work). Since there are now two worker, both waiting 10 seconds the total time goes down to 10 seconds.

---

wrong

Solution 2

Assuming you run your Django-server just with run() , by default this makes a single threaded server. If you use sleep on a single threaded process, the whole application freezes for that sleep time.

Solution 3

It may simply be that your browser is queuing the second request to be performed only after the first one completes. If you are opening your URLs in the same browser, try using the two different ones (e.g. Firefox and Chrome), or try performing requests from the command line using wget or curl instead.

Comments

[user984003]

So I just can't use sleep() because this will affect every call?? Is this just a Django issue? I do need some way to have my functions pause without every call being paused.

[user984003]

BTW, not sure if using Django automatically means that I am running the Django server. Webfaction uses Apache.

[Gjordis]

Good point, still the effect seems the same. Only one thread handles both requests, if both of them hang

[user984003]

Thanks for looking into this. Even without sleep, there seems a lot of reasons for why I would NOT want calls to run in the same thread. For example, if a call is looking up info from another website. urlopen() can be slow waiting for a response and I'd sure hate for every call to be waiting in a row for the other calls to finish. Or a slow database query. I will ask another question about how to start each call in a new thread.

[Jack Shedd]

Django is single-threaded, so that's always gonna be a problem. If you need asynchronous, non-blocking stuff, you'll probably want to look at something like Tornado.