1

Running some tests right now and it took 16min to process just 2600 accountshare upserts. With a 

Timeout : Took more than five minutes to process 200 Trying again later.

error. Which seems quite slow. I know using bulk api should mean you don't need the results to be immediate, but I still would love to have a good idea of the max time it takes for this,to account for scheduling purposes?

Will be sending salesforce hundreds of bulk jobs every time my program is run of various sizes (0-1000s records depending on if things need to be changed). Just seems hard to predict this stuff without having to have a thread watching every single job for a response.

How have y'all utilized bulk api in applications that run on a schedule?

|improve this question
  • 1
    If you're using the bulk API for "zero to thousands" of records, you're using it wrong. You should only use it when you're very nearly at the 10,000 row limit. – sfdcfox Apr 3 '17 at 21:49
  • In the past we were using normal soap calls but we ended up encountering all kind of time out issues and what not when automated. At the very least bulk api will eventually get things done.... i think? – J. Zhang Apr 4 '17 at 13:15
3

PK Chunking is always a good option when you know you're going to be dealing with lots (100k - Millions) of records.

https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/async_api_headers_enable_pk_chunking.htm

But, it's only good for certain objects. With bulk API being asynchronous, it's very hard to give any guarantees on SLA for completion. I would suggest scheduling and/or sending your job request to off peak hours in relation to when production users are on the platform. If you aren't looking for results immediately I'd be fairly confident in saying they'd be good to go by morning.

|improve this answer
  • When you include a question in your answer, it is highly likely it will be interpreted as an attempt to ask a new question. I've removed your last line for that reason. If you can rephrase it as a suggestion, it might be more appropriate to include in an answer. – Adrian Larson♦ Apr 4 '17 at 2:06
  • Thanks @AdrianLarson! Will consider this with future responses. – Brand0R Apr 4 '17 at 2:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.