1

Not sure if this is the right place for this, but figured I'd try:

Our Bulk Data Load jobs have been taking a really long time to complete recently. The processing time is normal, it's all queue time that's causing the delay. (Eg. 10:27 to process 25 records, only 1392ms of which was processing time.)

I'm looking at the data in Monitoring > Bulk Data Load Jobs, but it isn't giving me any transparency into what is causing the hold up.

We've started using more Bulk Data recently, and we've implemented two new packages (Xactly & Pardot) that heavily rely on apex jobs & API calls to integrate, so I'm inclined to think there's something on my end and this isn't just NA39 being slow, but I'm not sure where to start.

Is there something I can log into or run on my end to get some more visibility in this?

4

The Bulk API operates like other asynchronous features, such as the Metadata API, Batchable classes, Sandbox refreshes, and so on. It goes in to a queue and is processed as resources allow. If you choose to use the Bulk API, you're effectively declaring that you wish to allow the server to process the records when it has available resources. This may take a few minutes depending on how many other Bulk API jobs are being processed as well. Synchronous tasks have priority over asynchronous tasks, so it is not uncommon to have to wait "a little while" for the process to finish. I agree that 10 minutes of wait time is a bit much, but not entirely unheard of. Use the Bulk API for mass updates when you don't need immediate processing, and use the other APIs, either SOAP or REST, when you need your data loaded faster. If you're using the Bulk API to load less than a few thousand records, you're probably using the wrong tool for whatever goal you're trying to accomplish.

  • We're using the Bulk API for this because the number of records could be 10k+ at a time, and we don't need the processing to be immediate. I originally gave you credit for the answer but that's bc you confirmed all my hunches. The question still stands if there's any way to see what other resources are blocking the queue, though. I'm presuming not all in one place since it's not just our resources in the queue? – mlpSFadmin Mar 22 '17 at 2:21
  • 1
    @mlpSFadmin You are correct. The Bulk API resources are shared across all clients on a pod, so only internal salesforce employees have access to this data (upper level support, product management, and developers/techs). You could always try to get a case in, but in most cases, they'll simply tell you "yes, that's how it works." – sfdcfox Mar 22 '17 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.