Encoding and Decoding Base64 Python / Salesforce

Question

I’m working on a small python script to export ContentVersion records from one Salesforce org and migrate them into another. I’m using simple-salesforce. I am able to export the ContentVersion and the relayed VersionData. I’m running into issues with figuring out what to do with the VersionData response and what the appropriate decide / encode process is to then resend the content version to the new org. The files are always corrupted and don’t open.

Here's sample code, note I'm doing this in a colab notebook:

#Query Content Files
#GET SOURCE NOTES
content_doc_source_data = source_sf.query_all("SELECT Id, ContentDocumentId, ContentLocation, ContentUrl, Description, FileExtension, FileType, IsLatest, OwnerId, PathOnClient, SharingOption, SharingPrivacy, Title, VersionData FROM ContentVersion WHERE FileType !='SNOTE'")
content_doc_source_data_df = pd.DataFrame(content_doc_source_data['records']).drop(columns='attributes')

content_doc_source_data_df

This returns a table, and you can see there's a path for the ContentDoc itself in VersionData.

So then I'm iterating through that table and querying the VersionData:

import json

headers = {
        'Authorization': 'Bearer %s' % access_token
    }

for index, row in content_doc_source_data_df.iterrows():
  r = requests.request('get', instance_url+'/services/data/v52.0/sobjects/ContentVersion/'+row['Id']+'/VersionData', headers=headers)
  content_doc_source_data_df.at[index, 'File Body'] = r.text
  break

display(content_doc_source_data_df)

I then add the file body to the table just so I can interact with it, see picture below:

I think my question more specifically is, what should I be returning from the response (e.g., text, content, etc.) and then what are the next steps to decode/encode so that I can then insert that file into another Salesforce org.

You can see "File Body" column above. I was looking for guidance on converting it to base64 string.

I'm specifically looking for guidance on the best way to "decode" the file body and then turn around and decode it so I can send it to another Salesforce org.

Answer 1

The reason you're getting those question-mark replacement characters is the content that's coming back to you isn't text - but you're asking Python / requests to interpret the content as if it were.

You need to work with this content as a binary response, using the r.content property on the response object. That will give you a bytes object, rather than a str.

Then, to insert arbitrary content that you have in a bytes as a new ContentVersion, you'd call the Insert ContentVersion endpoint by using requests's multipart-encoded request support, something like this (untested code, written here!):

def upload_content_version(
    file_name: str,
    content: bytes,
    instance_url: str,
    access_token: str
) -> Response:
    return requests.post( 
        f"{instance_url}/services/data/v56.0/sobjects/ContentVersion",
        data={"PathOnClient": file_name},
        headers={"Authorization": f"Bearer {access_token}"},
        files={
            "file": (filename, content, "application/octet-stream")
        }
    );