Python del statement

Question

Calling del on a variable in Python. Does this free the allocated memory immediately or still waiting for garbage collector to collect? Like in java, explicitly calling del has no effect on when the memory will be freed.

Solution 1

The del statement doesn't reclaim memory. It removes a reference, which decrements the reference count on the value. If the count is zero, the memory can be reclaimed. CPython will reclaim the memory immediately, there's no need to wait for the garbage collector to run.

In fact, the garbage collector is only needed for reclaiming cyclic structures.

As Waleed Khan says in his comment, Python memory management just works, you don't have to worry about it.

Solution 2

"Deletion of a name removes the binding of that name from the local or global namespace". No more, no less. It does nothing to the object the name pointed to, except decrementing its refcount, and if refcount is not zero, the object will not be collected even when GC runs.

Solution 3

Also, the del statement seems to be a little bit faster than assigning None (similar to Java's style assigning null to a variable to free its memory ...).

To compare:

import time, math

def measure_del():
        start = time.time()
        for i in range(0,int(math.pow(10,8))):
                    a = "123"
                    del a # <--- !!!
        end = time.time()
        print(end-start)

def measure_none():
        start = time.time()
        for i in range(0,int(math.pow(10,8))):
                    a = "123"
                    a = None # <--- !!!
        end = time.time()
        print(end-start)

results in (running in idle3.4):

>>> measure_del()
3.9930295944213867
>>> measure_del()
3.7402305603027344
>>> measure_del()
3.8423104286193848
>>> measure_del()
3.753770351409912
>>> measure_del()
3.7772741317749023
>>> measure_del()
3.815058946609497

>>> measure_none()
4.052351236343384
>>> measure_none()
4.130320072174072
>>> measure_none()
4.082390069961548
>>> measure_none()
4.100180625915527
>>> measure_none()
4.071730375289917
>>> measure_none()
4.136169672012329

Comments

[Waleed Khan]

The point of garbage collection is that you don't have to worry about when the memory is freed. So why are you worrying about it?

[totoromeow]

I'm processing large volume of traffic and observed memory leak issues. I'm almost certain the issue is not in the python script, but just trying to make sure.

[martineau]

I don't believe CPython ever frees the memory back to the OS, even after it's been reclaimed.

[totoromeow]

So when you say "If the count is zero, the memory can be reclaimed.", do you mean that it still needs to wait for garbage collector to be "actually" reclaimed or?

[yantrab]

I'm not sure how else to say it: "CPython will reclaim the memory immediately, there's no need to wait for the garbage collector to run."

[Pavel Komarov]

That comment is in contradiction with a lot of what I've seen around SO today, but reading a large database table in to memory as a copy and then del reference causes the memory to be freed, so garbage collection is getting called, and (though there are no time guarantees on this) the interpreter is giving that memory back to the system. If I instead read my table and don't assign it to a variable, then it is an island, and I have to call gc.collect() explicitly to find it and free.

[yantrab]

@pvlkmrv That doesn't sound right: If you don't assign the table to a variable, then the reference count will be zero, and it will be reclaimed. Islands aren't a problem, it's circular references that require a gc pass to reclaim.

[Pavel Komarov]

Empirical observation shows that creating a large list in-line and passing it as an indexing object to a numpy array causes that list to hang around in memory even after I del the reference to the returned sub-array and Python decides in its own time (usually a second or so) to give that memory back to the system. Passing a numpy array as the indexing object, I observe the memory of that object is freed after the sub-array is returned.

[Pavel Komarov]

Likewise however I read a table (in a terminal, mind you), if I don't assign it to a variable, it does take memory and stick there even though I have no reference to it.

[yantrab]

If you do the same thing twice: A) assigning the result to a variable, and B) not assigning it to a variable, there is no way that case B will take longer to reclaim the memory than case A will. I'm not sure what you are observing, but the delay is not because the value isn't assigned to a variable.

[Pavel Komarov]

The bottom line is that, del variable seems to cascade in to a garbage collection of that object rather immediately if the refcount goes to zero, and in cases where the interpreter has malloced large sections of memory with mmap rather than on the heap with brk/sbrk it seems to return that memory to the system fairly immediately too.

[yantrab]

del x doesn't do anything different for garbage collection than x = None does. It just removes a reference from x's value.

[yantrab]

Why are programmers so fascinated with the speed of things that make no difference to the overall speed of the program? :)

[Barpfotenbaer]

Well, it's something like a pack of paper. E.g. you can buy 500 sheets with a grammage of 80g/m² or 90g/m². If you write on one single sheet, you'll hardly notice a difference. But if you compare both packages, you'll see a clear difference in height, weight and price. Writing good code, it is certainly not a bad idea to balance the readability of the code with the performance on machine level.

[Olivier RD]

In some cases you want that speed, because you need to free some underlying C++ memory before running out of buffer for example. The exact case that made me read that question.