Unicode issue in Python

Question

I am trying to download SFDC data using the REST API and Python, using simple_salesforce. So I am not sure if this is a SFDC question or a Python question. And I am aware of Python unicode support and issues, but not sure how that helps me with this problem.

I am having a problem handling UTF-8 encoding. Some of the data I retrieve is UTF-8 encoded, such as the "not equal" symbol with the actual slash thru the equals sign. This is \u2260.

Once I have retrieved the data from SFDC, I am trying to write it to a CSV file. Before I ran into the Unicode character, I simply wrote out the value. In the example below, when 'rec[field]' is the value 'False', then Python interprets the field as False, and says that

      'bool' object has no attribute 'encode'

If I wrap it in str(), that works, but then the error becomes:

 'ascii' codec can't encode character u'\u2260' in position 0: ordinal not in range(128)


                    fieldData1 = rec[field].encode("utf-8")

                    fieldData2 = str(rec[field]).encode("utf-8")

                    rowData.append(fieldDatax)

So this is why I am not sure whether this is a Python issue or not. I would prefer to just try and use the encode method all the time, but Python interprets the string 'False' as Python False.

It seems like it really should not be this hard to simply take whatever I receive from SF and write it out as a UTF-8 encoded csv file.

Any suggestions will be greatly appreciated.

Answer 1

This sounds like an issue purely on the Python side. It's one I've run into in some form or another many times in Python whether or not I was working with Salesforce data (usually via simple-salesforce which is wonderful).

I think what you want to do is wrap in unicode() rather than str().