0

I am using the simple_salesforce Python package. I tried bulk inserting 100 records using bulk.insert(), and did so without any errors. However, when I checked how many records were present after, there were only 19.

I'm curious if there is an insert limit I'm unaware of. Let me know what you think!

EDIT: Sorry! Forgot to show my code:

new_data = [{'Name': r['name'], 'N_Transactions__c': r['transactions']} for i, r in gmvs.iterrows()]
sf.bulk.GMVS__c.insert(new_data)

... where r['name'] is a string and r['transactions'] is an integer

  • 2
    Please edit to show the code you're using, and what the results look like when you review the bulk data job in Setup. – David Reed Sep 16 '19 at 18:01
  • 1
    more likely than not, 81 of the Accounts failed some validation rule (simple_salesforce may be configured with DmlOptions AllOrNothing=false) – cropredy Sep 16 '19 at 20:19
  • @cropredy the thing is, I tried inserting 10 at a time and was able to successfully insert all records, so I don't think its an issue with particular records – Tim Sauchuk Sep 17 '19 at 16:22
  • @DavidReed sorry! just added a code snippet to the post – Tim Sauchuk Sep 17 '19 at 16:27
  • @TimSauchuk You really need to be looking at the bulk job results in Setup. That's what is going to tell you what happened on a row-by-row basis. cropredy is almost certainly right that 81 records just failed for some reason. – David Reed Sep 17 '19 at 16:28
2

The Bulk API is not an "all or none" proposition, like regular DML may be. When records fail within a Bulk job, others may succeed and be persisted to the environment. The actual Bulk API job limits are measured in gigabytes; 100 records won't strain it at all.

What appears to be happening here is that many of your records are simply failing to insert, for reasons we can't tell right here. To determine the causes, you need to review the row-level success or failure information.

In Lightning Setup, navigate to Environments->Jobs->Bulk Data Jobs. You should see a listing for your job (probably more than one) under "Completed last 7 days". There, you can download the result file, which will give you row-by-row success and error details.

| improve this answer | |
  • Thanks David! I saw that all of my failures had the same error message: DUPLICATE_VALUE: Maximum number of duplicate updates in one batch (12 allowed). Attempt to update Id more than once in this Api call My questions is: How am I duplicating the Id field if I don't use it for an insert? Not sure how to not duplicate it... – Tim Sauchuk Sep 17 '19 at 18:07
  • @TimSauchuk Well, that points to a core issue in your Process automation; you may want to open a new question for that. – David Reed Sep 17 '19 at 18:08
  • sorry, updated my comment. Looks like the "Id" field was duplicated, though I'm not sure how that would happen since the Id field is generated automatically upon each insert. @David Reed – Tim Sauchuk Sep 17 '19 at 18:12
  • @TimSauchuk Something in your automation tried to update the same record more than once in a single DML operation. I couldn't say what that is without a lot more detail, but it's not per se related to your Bulk API work. – David Reed Sep 17 '19 at 18:13
  • gotcha, i'll look through my code to see if I tried updating an existing Id @David Reed – Tim Sauchuk Sep 17 '19 at 18:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.