2

I'm using the Bulk API to make updates to ~50k records. I'm splitting those up into jobs of about 5k apiece, but I was hoping there'd be a way to control the batch size similar to that of using Dataloader (ie: you can tell it to process in chunks of 200 records at a time, or 1 record at a time).

Is this configurable? Or if not is there a different way to achieve this? The main issue is that some of the jobs fail due to a record triggering too many workflows that bog down the execution time, so I'd rather try and isolate those and use smaller batch sizes.

3

The Bulk API is intended to load large numbers of records in parallel asynchronously. It also has really small daily limits. You could technically set up 10,000 batches of 1 record each, but then you'd hit your daily limit. Use the normal synchronous API instead. If you need batches of smaller than about 1,000 records, the Bulk API is not for you. Please note that, unlike the synchronous API, each batch has a limit of 10 minutes (with retries for additional chunks), and each chunk of 200 records has 5 minutes, instead of the usual smaller time limits associated with synchronous updates.

With the synchronous API, the batch size is how many records are submitted at once per API call. In the Bulk API, the batch size is how many records are submitted in a file, which is termed a batch. The batch is then broken down in to chunks of 100/200 records each (depending on API version). As long as each chunk runs in less than 5 minutes, you should be okay. If you're not able to get decent performance with values smaller than 1,000 or so, it's simply going to be too "expensive" in terms of daily limits to use this API.

  • I think I understand, but want to confirm: essentially Bulk isn't going to work if I want to do very small batches, and I should instead take those records and use something like this: developer.salesforce.com/docs/atlas.en-us.216.0.api_rest.meta/… – Matt Oct 14 '18 at 6:04
  • 1
    @Matt or the SOAP API. Data Loader in non-bulk mode uses this API if you don't want to roll your own solution. – sfdcfox Oct 14 '18 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.