5

we have a process where we are loading events in via the Data Loader and have a before insert trigger on the event/activity to handle these records as they are being processed. We'd like that if one of the records has an error to fail the entire job, instead of just failing that record and allowing a partial insert.

I've done some digging and unfortunately it seems like this isn't possible due to the batch nature of using the data loader and the bulk api, but hoping I'm missing something. Thanks!

  • 1
    I looked into the Data Loader settings for this feature. Unfortunately, did not able to find anything for this. May be this feature is not available at this moment. – Devendra Aug 11 '16 at 18:05
1

There is really no way to do this because of the way Data Loader works

For example - imagine Data Loader batch size is 4 and you have these records

00Event
01Event
02Event
03Event
---- transaction boundary ----
04Event
05Event

So, further imagine that all six (6) events belong logically to the same parent object. And that 04Event fails validation rules and hence all six events should be rolled back

But, the first 4 events are in one transaction where all records pass VRs and hence are committed - even if data loader didn't support partial successes (which it does). Even if records [4] and [5] failed to insert as a batch, your parent object would have an incomplete list of events -- at least as far as your application requirements seem to demand

So, a workaround solution is to

  • first update the parent object with a count of expected children events,
  • upload the events per Data Loader with partial success supported
  • have a RSF on the parent object with a count of actual Events
  • an IMAGE formula on your detail page (or a workflow to notify sysad) that indicates that one or more events are missing when expected != actual count of child events.
0

I'd be curious if instead of rolling back you could try to pre-validate the data by loading it in a test method via Test.loadData().

The assumption is that if the data can be loaded in a test, it will also be successfully loaded by the data loader.

The caveats are:

  1. You have Apex skills

  2. The CSV files that contain KBs and not MBs of data. Tests will timeout loading large amounts of data in a test.

  • This approach would also be subject to governor limits, so you'd top out at 10K records total. – Adrian Larson Dec 7 '16 at 0:40
  • @Adrian I'm somewhat new to Salesforce, is the 10K record limit for the single test invocation? It would take some extra work to chunk large data sets and for the logic to call the test repeatedly until all data is validated. Of course, you may run into other limits like CPU with this approach. – Arpi Jakab Dec 7 '16 at 22:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.