Get a mapping of External Ids and Salesforce Ids when migrating data from simple-salesforce

Question

I want to migrate Account data into Salesforce and create a lookup table containing Salesforce IDs and External IDs.

I am using the simple-salesforce library to perform a bulk insert of Accounts into Salesforce.

bulk_result = sf.bulk.Account.insert(bulk_data)

My table has a column ExternalId__c. But Salesforce returns the migration results in a specific format, which does not contain External IDs, eg.

[{
    "success": true,
    "created": true,
    "id": "001280000000YXSAA3",
    "errors": []
},
{
    "success": false,
    "created": false,
    "id": null,
    "errors": [
        {
            "statusCode": "FIELD_INTEGRITY_EXCEPTION",
            "message": "Error Message",
            "fields": [
                "Phone"
            ]
        }
    ]
}]

I have noticed that the results produced are in the same order in which the data is loaded, this observation could be incorrect as I've migrated only a few records. My concern is that, in the future when I want to migrate thousands of records, maybe parallelly, this approach for mapping IDs could fail.

I've previously used DBAmp to migrate data which accurately creates the lookup tables. So, is there a way the same can be achieved from Python? Is there an alternate library or function I can use?

My current approach is:

def data_migration_to_sf(object_name, df, sf):

  # Convert dataframe 
  df_list = df.to_dict('records')

  # Insert data into Salesforce
  results = eval(f'sf.bulk.{object_name}.insert(df_list)')

  # Initialize error list
  error_list = []
  sf_ids = []

  for result in results:
    if result["errors"]:
      error = result["errors"][0]
      formatted_error = f'{error["statusCode"]}:{error["message"]}; "Fields":{",".join(error["fields"])}'
      error_list.append(formatted_error)
    else:
      error_list.append("Operation Successful.")
      
    sf_ids.append(result['id'])

  # Update dataframe  
  df = df.drop('id', axis=1) 
  df.insert(0, 'Id', sf_ids)
  df['Error'] = error_list

  return df

Answer 1

No matter how you load the data, the API always returns results in the same order as the source API call. In other words, input[0] will always be aligned with output[0], and so on. You don't really need to concern yourself with the order in which the records are loaded.