How do I debug this Nginx to uWSGI timeout?

Question

I'm running a database-backed web site that receives very little traffic. However, once or twice day, a request will timeout and I'll see this (or a similar) error in Nginx's error.log:

2013/06/13 18:32:40 [error] 16723#0: *27796 upstream timed out (110: Connection timed out)
while reading response header from upstream, client: 199.71.215.214, server:
app.mypythonwebapp.com, request: "POST /api?submit_staker_response HTTP/1.1", upstream:
"uwsgi://unix:/var/run/uwsgi/app.mypythonwebapp.com-uwsgi.sock", host:
"app.mypythonwebapp.com", referrer:
"https://app.mypythonwebapp.com/survey/5/791/70ea73eb9a489f2dead804a95c400ab2"

I'm running uWSGI and there's nothing related to this at all (that I can tell) in its log file. I suspected it might be PostgreSQL related, but if I check its status via pg_stat_activity I see nothing out of the ordinary.

This is my uWSGI YAML config file:

uwsgi:
    socket: /var/run/uwsgi/%n-uwsgi.sock
    workers: 5
    buffer-size: 32768
    callable: app
    wsgi-file: /opt/sites/app.mypythonwebapp.com/run.py
    virtualenv: /opt/virtualenv/app.mypythonwebapp.com
    pythonpath: /opt/sites/app.mypythonwebapp.com

The server I'm on has two (virtualized) cores, so I did 1 + cores*2 to determine the number of workers. I also upped the buffer-size parameter to try to fix this but the error still occurs.

I'm not sure where to begin to debug this. I have little experience running uWSGI (or any Python WSGI implementation).

Solution

The option triggering the timeout (in nginx) is

http://wiki.nginx.org/HttpUwsgiModule#uwsgi_read_timeout

its default is 60 seconds, so if you request does not generate output in that timeslice nginx will close the connection.

If you do not see error in uWSGI (a part i suppose from the "broken pipe" as nginx disconnected) i would investigate why the generation is so slow

Comments

[skyler]

I can't replicate this on my local computer, and I don't want to profile production requests. How would you investigate why the generation is so slow? Should I profile the requests?