How to prevent 'IOError: failed to write data' when client closes connection to Django/WSGI app?

Question

I have an iPhone app that is using web services implemented in Python, using Django and Piston, running on an apache server through WSGI.

Sometimes the app closes its connection to the server before a call is finished. When it does this it causes a:

[Tue Sep 06 11:29:46 2011] [error] [client 207.35.164.99] mod_wsgi (pid=820): Exception occurred processing WSGI script 'myscript.wsgi'.
[Tue Sep 06 11:29:46 2011] [error] [client 207.35.164.99] IOError: failed to write data

to appear in my server's error logs.

I can "fix" the problem in the app by not explicitly closing the connection, but just leaving it to finish downloading and ignoring the result. However, I'd like fix this on the server-side if possible. How can I do it?

Solution 1

[disclaimer: this is a "why it can't be easily done" explanation, not a solution]

As @Slott pointed out, this is definitely the technically correct behavior when stream.close or stream.write is called on a closed socket. However, I understand the motivation for the question... in the context of a wsgi app, clients terminating the connection after a full or partial read is not an "exceptional" behavior, it happens all the time. For it to be left unhandled leaves the impression it was unexpected / the code was unprepared for this, when in fact it's expected, and shouldn't be worthy of note. So it'd be nice to fix.

The catch is that you'd have to find a way to distinguish cases...

• Situations like "client read 'Status: 304', and then closed connection" or "client read all bytes, and then closed connection, even though it had requested connection should be reused" are ones where it would be appropriate to not issue any sort of logging besides a log.debug() call.

• But situations like "client stopped reading in the middle of file because connection died when an ISP router had a stroke" are worthy of an error being logged. Something didn't successfully complete, and any transactional state your server app builtup should be rolled back. In which case IOError propagating upwards is the right thing to do.

Such errors are only silenceable if at every place they could be raised, the code is modified to distinguish those two cases. Until then, the wsgi authors seem to have erred on the side of caution. So there isn't a quick fix for this that I know of.

---

(Also, I should note this isn't django-specific, I use paste+pylons and have the same thing happen)

Solution 2

See this: http://code.google.com/p/modwsgi/issues/detail?id=29

If processing iterable, then message logged a debug level and no Python exception used. Thus to see client closed connection problems when iterable used, then need LogLevel to be debug.

Apparently, you would need to tweak your Django response to be an iterable instead of a string.

Comments

[Josh Knauer]

Thanks for the tip. The exception seems to be coming from deep in the guts of piston (a python library I'm using), so a fix would probably need to be there. I'll take a stab at it.

[user1066101]

@Josh Knauer: It would be far, far easier to simply fix your app than to mess around trying to silence a normal message that indicates your app is not following the HTTP rules.

[Josh Knauer]

I agree. As it is, behaviour in app is already fixed. My interest in digging into piston is probably more so for me learn more about the library as opposed to me thinking this is the best idea for the general case :) I suppose in the context of this question though it would be better to leave the answer at "It's normal, just leave it.'

[Nimeshka Srimal]

but what if the images are not uploading? I'm also having this problem, it works locally, but not in the server.