What does x[x < 2] = 0 mean in Python?
Solution 1
This only makes sense with NumPy arrays. The behavior with lists is useless, and specific to Python 2 (not Python 3). You may want to double-check if the original object was indeed a NumPy array (see further below) and not a list.
But in your code here, x is a simple list.
Since
x < 2
is False i.e 0, therefore
x[x<2]
is x[0]
x[0]
gets changed.
Conversely, x[x>2]
is x[True]
or x[1]
So, x[1]
gets changed.
Why does this happen?
The rules for comparison are:
When you order two strings or two numeric types the ordering is done in the expected way (lexicographic ordering for string, numeric ordering for integers).
When you order a numeric and a non-numeric type, the numeric type comes first.
When you order two incompatible types where neither is numeric, they are ordered by the alphabetical order of their typenames:
So, we have the following order
numeric < list < string < tuple
See the accepted answer for How does Python compare string and int?.
If x is a NumPy array, then the syntax makes more sense because of boolean array indexing. In that case, x < 2
isn't a boolean at all; it's an array of booleans representing whether each element of x
was less than 2. x[x < 2] = 0
then selects the elements of x
that were less than 2 and sets those cells to 0. See Indexing.
>>> x = np.array([1., -1., -2., 3])
>>> x < 0
array([False, True, True, False], dtype=bool)
>>> x[x < 0] += 20 # All elements < 0 get increased by 20
>>> x
array([ 1., 19., 18., 3.]) # Only elements < 0 are affected
Solution 2
>>> x = [1,2,3,4,5]
>>> x<2
False
>>> x[False]
1
>>> x[True]
2
The bool is simply converted to an integer. The index is either 0 or 1.
Solution 3
The original code in your question works only in Python 2. If x
is a list
in Python 2, the comparison x < y
is False
if y
is an int
eger. This is because it does not make sense to compare a list with an integer. However in Python 2, if the operands are not comparable, the comparison is based in CPython on the alphabetical ordering of the names of the types; additionally all numbers come first in mixed-type comparisons. This is not even spelled out in the documentation of CPython 2, and different Python 2 implementations could give different results. That is [1, 2, 3, 4, 5] < 2
evaluates to False
because 2
is a number and thus "smaller" than a list
in CPython. This mixed comparison was eventually deemed to be too obscure a feature, and was removed in Python 3.0.
Now, the result of <
is a bool
; and bool
is a subclass of int
:
>>> isinstance(False, int)
True
>>> isinstance(True, int)
True
>>> False == 0
True
>>> True == 1
True
>>> False + 5
5
>>> True + 5
6
So basically you're taking the element 0 or 1 depending on whether the comparison is true or false.
If you try the code above in Python 3, you will get TypeError: unorderable types: list() < int()
due to a change in Python 3.0:
Ordering Comparisons
Python 3.0 has simplified the rules for ordering comparisons:
The ordering comparison operators (
<
,<=
,>=
,>
) raise aTypeError
exception when the operands don’t have a meaningful natural ordering. Thus, expressions like1 < ''
,0 > None
orlen <= len
are no longer valid, and e.g.None < None
raisesTypeError
instead of returningFalse
. A corollary is that sorting a heterogeneous list no longer makes sense – all the elements must be comparable to each other. Note that this does not apply to the==
and!=
operators: objects of different incomparable types always compare unequal to each other.
There are many datatypes that overload the comparison operators to do something different (dataframes from pandas, numpy's arrays). If the code that you were using did something else, it was because x
was not a list
, but an instance of some other class with operator <
overridden to return a value that is not a bool
; and this value was then handled specially by x[]
(aka __getitem__
/__setitem__
)
Solution 4
This has one more use: code golf. Code golf is the art of writing programs that solve some problem in as few source code bytes as possible.
return(a,b)[c<d]
is roughly equivalent to
if c < d:
return b
else:
return a
except that both a and b are evaluated in the first version, but not in the second version.
c<d
evaluates to True
or False
.
(a, b)
is a tuple.
Indexing on a tuple works like indexing on a list: (3,5)[1]
== 5
.
True
is equal to 1
and False
is equal to 0
.
(a,b)[c<d]
(a,b)[True]
(a,b)[1]
b
or for False
:
(a,b)[c<d]
(a,b)[False]
(a,b)[0]
a
There's a good list on the stack exchange network of many nasty things you can do to python in order to save a few bytes. https://codegolf.stackexchange.com/questions/54/tips-for-golfing-in-python
Although in normal code this should never be used, and in your case it would mean that x
acts both as something that can be compared to an integer and as a container that supports slicing, which is a very unusual combination. It's probably Numpy code, as others have pointed out.
Solution 5
In general it could mean anything. It was already explained what it means if x
is a list
or numpy.ndarray
but in general it only depends on how the comparison operators (<
, >
, ...) and also how the get/set-item ([...]
-syntax) are implemented.
x.__getitem__(x.__lt__(2)) # this is what x[x < 2] means!
x.__setitem__(x.__lt__(2), 0) # this is what x[x < 2] = 0 means!
Because:
-
x < value
is equivalent tox.__lt__(value)
-
x[value]
is (roughly) equivalent tox.__getitem__(value)
-
x[value] = othervalue
is (also roughly) equivalent tox.__setitem__(value, othervalue)
.
This can be customized to do anything you want. Just as an example (mimics a bit numpys-boolean indexing):
class Test:
def __init__(self, value):
self.value = value
def __lt__(self, other):
# You could do anything in here. For example create a new list indicating if that
# element is less than the other value
res = [item < other for item in self.value]
return self.__class__(res)
def __repr__(self):
return '{0} ({1})'.format(self.__class__.__name__, self.value)
def __getitem__(self, item):
# If you index with an instance of this class use "boolean-indexing"
if isinstance(item, Test):
res = self.__class__([i for i, index in zip(self.value, item) if index])
return res
# Something else was given just try to use it on the value
return self.value[item]
def __setitem__(self, item, value):
if isinstance(item, Test):
self.value = [i if not index else value for i, index in zip(self.value, item)]
else:
self.value[item] = value
So now let's see what happens if you use it:
>>> a = Test([1,2,3])
>>> a
Test ([1, 2, 3])
>>> a < 2 # calls __lt__
Test ([True, False, False])
>>> a[Test([True, False, False])] # calls __getitem__
Test ([1])
>>> a[a < 2] # or short form
Test ([1])
>>> a[a < 2] = 0 # calls __setitem__
>>> a
Test ([0, 2, 3])
Notice this is just one possibility. You are free to implement almost everything you want.
aberger
Updated on July 26, 2022Comments
-
aberger almost 2 years
I came across some code with a line similar to
x[x<2]=0
Playing around with variations, I am still stuck on what this syntax does.
Examples:
>>> x = [1,2,3,4,5] >>> x[x<2] 1 >>> x[x<3] 1 >>> x[x>2] 2 >>> x[x<2]=0 >>> x [0, 2, 3, 4, 5]
-
matth about 8 yearsYou might mention that
x
and2
are "ordered consistently but arbitrarily" and that the ordering might change in different Python implementations. -
Karoly Horvath about 8 yearsYepp. python3: ideone.com/1Cw4gb and stackoverflow.com/questions/3270680/…
-
kratenko about 8 yearsI would also add that this is a clever way of doing things and should in my opinion be avoided. Do it explicitly - the fact that OP had to ask this question supports my point.
-
Iłya Bursov about 8 yearscan you add more details, why
x<2 == false
? -
Antti Haapala -- Слава Україні about 8 years
bool
is not converted to an integer, abool
in Python is an integer -
J Richard Snape about 8 yearsGiven that the OP specifically says "I came across some code like this...", I think your answer describing numpy boolean indexing is very useful - might be worth pointing out that if the OP scrolls up the code they looked at, they'll almost certainly see an
import
for numpy. -
cat about 8 years
+False
Hi Perl, hey JavaScript, how y'all doing? -
Antti Haapala -- Слава Україні about 8 years@cat in Javascript, Perl, it converts the value as number. In Python it is for
UNARY_POSITIVE
opcode that calls the__pos__
-
cat about 8 years
Code Golf is the art of writing programs
:') -
cat about 8 yearsMinor nitpick: The bool is not cast to an int, it just is one (see the other answers)
-
MSeifert about 8 yearsI think you meant
__setitem__
instead of__getitem__
in your last section. Also I hope you don't mind that my answer was inspired by that part of your answer. -
Antti Haapala -- Слава Україні about 8 yearsNo, I meant and was thinking of
__getitem__
though equally could have been__setitem__
and__delitem__
-
Tim Pederick about 8 yearsStill an overly clever way to do it, surely? (As compared with, say,
[0 if i < 2 else i for i in x]
.) Or is this encouraged style in Numpy? -
user2357112 about 8 years@TimPederick: Using list comprehensions with NumPy is a pretty bad idea. It's dozens to hundreds of times slower, it doesn't work with arbitrary-dimensional arrays, it's easier to get the element types screwed up, and it creates a list instead of an array. Boolean array indexing is completely normal and expected in NumPy.
-
PascalVKooten about 8 yearsI would say using anything really is way too general for logically explainable behavior like the accepted answer.
-
MSeifert about 8 years@PascalvKooten Do you disagree with the "anything" or with the generalized answer? I think it's an important point to make because most logical behaviour in python is just by convention.
-
porglezomp about 8 yearsJust to clarify @AnttiHaapala's statement for anyone else that comes along,
bool
is a subclass ofint
. -
Michael Delgado about 8 years@TimPederick In addition to the performance hit it's also likely that whoever wrote the code intended to keep using a numpy array.
x[x<2]
will return a numpy array, whereas[0 if i<2 else i for i in x]
returns a list. This is becausex[x<2]
is an indexing operation (referred to in numpy/scipy/pandas as a slicing operation due to the ability to mask data), whereas the list comprehension is a new object definition. See NumPy indexing