Deleting list elements based on condition

Question

I have a list of lists: [word, good freq, bad freq, change_status]

list_1 = [['good',100, 20, 0.2],['bad', 10, 0, 0.0],['change', 1, 2, 2]]

I would like to delete from the list all elements which don't satisfy a condition.

So if change_status > 0.3 and bad_freq < 5 then I would like to delete that the elements corresponding to it.

So the list_1 would be modified as,

list_1 = [['good',100, 20, 0.2],['bad', 10, 0, 0.0]]

How do I selective do that?

Solution 1

list_1 = [['good',100, 20, 0.2],['bad', 10, 0, 0.0],['change', 1, 2, 2]]
list_1 = [item for item in list_1 if item[2] >= 5 or item[3] >= 0.3]

You can also use if not (item[2] < 5 and item[3] < 0.3) for the condition if you want.

Solution 2

Use the filter function with an appropriate function.

list_1 = filter(lambda x: x[3] <= 0.3 and x[2] < 5, list_1)

---

Demo:

In [1]: list_1 = [['good',100, 20, 0.2],['bad', 10, 0, 0.0],['change', 1, 2, 2]]
In [2]: filter(lambda x: x[3] <= 0.3 and x[2] < 5, list_1)
Out[2]: [['bad', 10, 0, 0.0]]

Note that good doesn't satisfy your condition (20 < 5 is false) even though you said so in your question!

---

If you have many elements you might want to use the equivalent function from itertools:

from itertools import ifilter
filtered = ifilter(lambda x: x[3] <= 0.3 and x[2] < 5, list_1)

Comments

[agf]

His edited question doesn't match that inequality, and only use filter on built in functions, otherwise use a list comprehension.

[ThiefMaster]

"and only use filter on built in functions" - any reason for that?

[agf]

Because list comprehensions read better than filter on a lambda, and also generally perform better. The overhead of the lambda is fairly high. If performance matters, and there is an appropriate built-in (often from the operator module), then filter on a built-in can sometimes be faster than the equivalent list comprehension.

[Petr Viktorin]

Much more pythonic than filter with a lambda.

[agf]

He wants to remove the elements that meet those conditions, you need to reverse them (I made the same mistake).

[agf]

@savinos See my comments to ThiefMaster's answer. List comprehensions are generally considered to be more Pythonic, especially when map or filter would require a lambda rather than just a built-in.

[Sreenikethan I]

Doesn't this generate a new list? Is there a way to do it in-place?

[agf]

@SreenikethanI You can, if you manually assign items to their new indexes in the list, and then truncate it to its new length. But in general that will be much slower than this (if the time it takes is significant at all), and the list has to be very large for the memory usage to be significant since it only contains references.

[agf]

@SreenikethanI def shrink(lst): new = 0 for item in lst: if item % 2: # example condition lst[new] = item new += 1 lst[new:] = []

[Sreenikethan I]

@agf that reminds me... would the del statement be effective, if we for-loop through the list in reverse? That way we need not worry about the index changing... something like for i in range(len(list_1))[::-1]: if condition: del list_1[i]

[agf]

@SreenikethanI The problem is the performance of each del can be slow, because (depending on the specific implementation) each one may cause the list to be resized down. That's why my example code basically only calls del once; the final line is equivalent to del lst[new:]. The list comprehension is still the way I'd do it.

[Sreenikethan I]

@agf Oh, that makes a lot of sense now. Thanks

[Benyamin Jafari - aGn]

itertools.ifilter() was removed in Python 3 as the built-in filter() function provides the same functionality now.