Django memory leak: possible causes?

Question

I've a Django application that every so often is getting into memory leak.

I am not using large data that could overload the memory, in fact the application 'eats' memory incrementally (in a week the memory goes from ~ 70 MB to 4GB), that is why I suspect the garbage collector is missing something, I am not sure though. Also, it seems as this increment is not dependant of the number of requests.

Obvious things like DEBUG=True, leaving open files, etc... no apply here.

I'm using uWSGI 2.0.3 (+ nginx) and Django 1.4.5

I could set up wsgi so that restart the server when the memory exceeds certain limit, but I wouldn't like to do that since that is not a solution really.

Are there any well know situations where the garbage collector "doesn't do its work properly"? Could it provide some code examples?

Is there any configuration of uWSGI + Django that could cause this?

Solution

I haven't found the exact stuff I'm looking for (each project is a world!), but following some clues and advices I managed to solve the issue. I share with you a few links that could help if you are facing a similar problem.

django memory leaks, part I, django memory leaks, part II and Finding and fixing memory leaks in Python

Some useful SO answers/questions: Which Python memory profiler is recommended?, Is there any working memory profiler for Python3, Python memory leaks and Python: Memory leak debugging

Update

pyuwsgimemhog is a new tool that helps to find out where the leak is.

Comments

[hughdbrown]

What's the evidence that there is a memory leak? And, we'll need to see some actual code.

[trinchet]

isn't it enough evidence the increcement from 400 MB to 4G ? What part of the code you need?

[hughdbrown]

Re: "What part of the code you need?" Pretend someone was asking you this question. What would help you answer? I am thinking that code and configuration would be a good start. Anything else and people are just guessing on your behalf.

[trinchet]

Got it @hughdbrown, but I couldn't to put the entire project code here and also this isn't the idea, as you can see my questions are flexibles: "Are there any well know situations...", "Is there any configuration of uWSGI + Django that could cause..." in order to get some well know situations/issues around memory leak in Django.