pandas - change df.index from float64 to unicode or string

Question

I want to change a dataframes' index (rows) from float64 to string or unicode.

I thought this would work but apparently not:

#check type
type(df.index)
'pandas.core.index.Float64Index'

#change type to unicode
if not isinstance(df.index, unicode):
    df.index = df.index.astype(unicode)

error message:

TypeError: Setting <class 'pandas.core.index.Float64Index'> dtype to anything other than float64 or object is not supported

Solution 1

You can do it that way:

# for Python 2
df.index = df.index.map(unicode) 

# for Python 3 (the unicode type does not exist and is replaced by str)
df.index = df.index.map(str)

As for why you would proceed differently from when you'd convert from int to float, that's a peculiarity of numpy (the library on which pandas is based).

Every numpy array has a dtype, which is basically the machine type of its elements : in that manner, numpy deals directly with native types, not with Python objects, which explains how it is so fast. So when you are changing the dtype from int64 to float64, numpy will cast each element in the C code.

There's also a special dtype : object, that will basically provide a pointer toward a Python object.

If you want strings, you thus have to use the object dtype. But using .astype(object) would not give you the answer you were looking for : it would instead create an index with object dtype, but put Python float objects inside.

Here, by using map, we convert the index to strings with the appropriate function: numpy gets the string objects and understand that the index has to have an object dtype, because that's the only dtype that can accomodate strings.

Solution 2

For python 3 and pandas 0.19 or latter versions, I found the following works fine for me

    # Python 3 (pandas 0.19 or latter versions)
    df.index.astype(str, copy = False)

Comments

[Shivam Gaur]

This doesn't work on Python 3.5. Do you have any idea why?

[Arthur]

The original poster was using Python 2. the unicode type does not exist anymore in Python 3, and the str type must be used instead (basically, what was called str in Python 2 is called bytes in Python 3, and unicode likewise became str). See this question for more information.

[PMcK]

I tried this in Python 3 and it did not change anything. I am trying to go change an index from Object to a String.

[AndreaCassioli]

@PMcK I got the same issue. Any success?

[Michel de Ruiter]

Sometimes df.index = df.index.astype(int) is needed instead of copy=False.

[VaM999]

@MicheldeRuiter Could you tell me when assignment is needed instead of copy=False?

[Michel de Ruiter]

@VaM999 I don't remember... :-(

[Olivierwa]

I had an issue when using copy=False and the type was np.uint64, The output would not be unsigned anymore

[smci]

This doesn't work for Python 3.8/pandas 1.3.4. Both with copy=True and False. Can you recheck and confirm which versions this works in?

[Admin]

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.