python 3.x - Importing csv using pd.read_csv - invalid start byte error

Question

I'm trying to import a csv file using:

data = pd.read_csv("filename.csv")

I get the following error: "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 2: invalid start byte".

The answer in this question: UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c might work, but I am not sure how to implement it (I can't comment on the answer because I don't have enough reputation yet).

Any help would be appreciated.

Edit: The issue seems to be linked to the fact that I have a degree symbol. It would be fine for me if during import this issue is just skipped.

Answer 1

If you face an encoding error due to encoding on your file not being the default as mentioned by the pd.read_csv() docs , you can find the encoding of the file by first installing chardet followed by the below code:

import chardet    
rawdata = open('D:\path\file.csv', 'rb').read()
result = chardet.detect(rawdata)
charenc = result['encoding']
print(charenc)

This will give you the encoding of the file.

Once you have the encoding, you can read as :

pd.read_csv('D:\path\file.csv',encoding = 'encoding you found')

or

pd.read_csv(r'D:pathfile.csv',encoding = 'encoding you found')

You will get the list of all encoding here

Hope you find this useful.