UnicodeDecodeError: 'cp932' codec can't decode byte 0xfc

Question

import os

for root, dirs, files in os.walk('Path'):
     for file in files:
         if file.endswith('.c'):
             with open(os.path.join(root, file)) as f:
                    for line in f:
                        if 'word' in line:
                            print(line)

getting the error

UnicodeDecodeError: 'cp932' codec can't decode byte 0xfc in position 6616: illegal multibyte sequence

I think file needs shift jis encoding. can i set encoding at start only? i tried setting with open(os.path.join(root, file),'r',encoding='cp932') as f: but got same error

Solution 1

You could pass errors='ignore', but make sure to check what is the encoding of your files.

open(os.path.join(root, file),'r', encoding='cp932', errors='ignore')

Solution 2

Ended up here because I got the same error.

I'm just learning, but fortunately I found a solution.

If it says:

UnicodeDecodeError: 'cp932' codec can't decode

it means that the file that you are using is not encoded in cp932, so you actually need to change the encoding.

In my case, I was trying to read a file encoded in UTF-8, so the solution was to include that when I opened my file:

open("file.txt","r",encoding='utf-8')

I hope that this helps anybody who comes here because of the same error.

Comments

[Chetan.B]

still same problem : for line in f: UnicodeDecodeError: 'cp932' codec can't decode byte 0xfc in position 6616: illegal multibyte sequence

[Chetan.B]

It will ignore error ans skip that file is it like that?

[cbodt]

It will not ignore the file completely, but just the characters that cannot be decoded inside the file. Maybe there only some files or lines incorrectly encoded. You could check how many of these errors you have by catching the exception and printing the filename.

[Žilvinas Rudžionis]

it is possible to share file content?

[Mr_and_Mrs_D]

And will corrupt your data @Chetan.B - terrible idea