This is a bit of an odd one. We are interacting with an external system that does not support concurrency due to a bug which they are struggling to fix. We perform a HTTP callout, the callout takes between 15 - 45 seconds to run.

During this time no other request to this web callout can occur or the system it is calling to will essentially crash. Because of how long this operation is, it must be an asynchronous job of some sort

NOTE: To clarify, if different transactions invoke this same callout within the same period it will cause the same issue, this problem is separate to Salesforce Transactions. A single Salesforce Transaction could require multiple callouts though

What we've thought of so far:

  • Use scheduled batches (size of 1) and remove any transactional interactions with the API: This is an okay solution, it does allow us to run each call one at a time relatively safely, but we would like something that is closer to real time if possible
  • Chained Jobs using Flex Queue + Queueable Interface: This sounds okay on a transactional level, but if two transactions occur at the same time / similar time it will create multiple chains
  • Future jobs: This is what we're using right now, Future jobs will run whenever regardless of the status of other jobs and definitely allow the issue to occur

Is there another type of asynchronous processing interface we could not think of? Or a Salesforce mechanism to guarantee only one of an Interface will run at any given time

  • 2
    You could create a queue object and instead of running the future write a record in the queue object. Then have a batch work through that one at a time
    – Eric
    Apr 30, 2018 at 4:39
  • That's the direction I was thinking with the Scheduled Batches. Not ideal, but it will work. I was hoping there was an alternative that might work
    – Mattisdada
    Apr 30, 2018 at 5:14
  • Not sure its a perfect way but can d one thing. Create a custom setting and check it when you are making callout and uncheck it after transaction complete. All callout if this checkbox is checked show user some information to try again. Not a best approach but will work. Apr 30, 2018 at 5:17
  • @Mattisdada the batch could run all the time continuously.
    – Eric
    Apr 30, 2018 at 11:19
  • @Eric An infinitely chained Queueable job? Otherwise I don't know how you could run a batch all the time
    – Mattisdada
    May 1, 2018 at 6:31

3 Answers 3


I've implemented the following in numerous orgs and it works pretty well. It's similar to Keith's suggestion, but a bit more detailed and it does usually do near-realtime processing:

  1. Create a custom object to queue records that need to be sent via callout. Each time a callout is required add a record, use an auto-number to preserve ordering
  2. Do all of your callouts in a Queueable which processes one record at a time
  3. Create another custom object for mutual exclusion (let's call it Mutex__c). This has a external id field on it, referring to the process you want to run (I often use this to manage multiple integrations in one org) and a checkbox field on it called something like Run_Queueable__c
  4. Create a trigger on Mutex__c object which starts a Queueable when Run_Queueable__c turns from false to true
  5. Have your Queuable set Run_Queueable__c to false when it has nothing left to process, otherwise keep re-queueing itself until everything is done
  6. Have a trigger on the queue items which does an upsert with the queueable name and Run_Queueable__c = true

This ensures that only one Queuable is running at once. Even if two transactions start at once, only one of them gets to set the mutual exclusion record from false to true. The other one just writes over the true value with true again, so it doesn't start another Queueable.

So, I'd have something like this as a trigger on the queue object:

Set<String> doCalloutStatuses = new Set<String> {

for(Integer i=0; i < newList.size(); i++) {
    My_Queue_Object__c newQ = newList[i];

            && (oldList == null
            || !doUpsertStatuses.contains(oldList[i].Callout_Status__c))) {
        upsert new Mutex__c(Queueable_Name = MyQueueable.class.getName(), Run_Queueable__c = true);

I can't really post all the code as it's integrated into a load of internal libraries that we have. But, hopefully, you get the idea.

Generally, it works well for me. The one major complication has been that if the Mutex__c object gets out of sync with what's actually running, then you're in trouble. This can happen during an org-split or SF maintenance where they kill your job before it has chance to set Run_Queueable__c=false. They you get what they call a zombie process in Unix, so you need a scheduled job to go reap them.

  • Is there a FOR UPDATE somewhere in the process that upserts the Mutex__c records? Couldn't there be a race condition with multiple transactions where the trigger sees Run_Queueable__c toggle to true? May 1, 2018 at 1:23
  • 1
    I don't have a FOR UPDATE in there. I don't think that there is a race condition because there is only one process allowed to set Run_Queueable__c to false (the Queueable itself). Multiple transactions trying to set it to true isn't a true race condition because it doesn't matter what the eventual order is. The DB makes one of them win, so it toggles exactly once and the rest of the order is irrelevant. This has been running in prod for years for many customers, including some very busy systems. I would love to write a blog about this one day, but it would be LONG.
    – Aidan
    May 1, 2018 at 10:48
  • Just for fun I tried to break it by introducing a delay in the trigger execution to lengthen the transaction time. Seems to handle it just fine. You either get the latest/current value for the field via Trigger.oldMap or the parallel transaction fails out with a UNABLE_TO_LOCK_ROW. ACID database++ May 2, 2018 at 0:55
  • That's one of the details in the real system. The mutex record become a global that everyone wants access to, and this can cause row lock errors. So, rather than do an upsert, I actually query it first (without FOR UPDATE or that would defeat the point) and only set if I'm actually going to change the value. And I catch row lock errors to do a few retries. In a B2C org where the customers can trigger callouts, I found this was necessary. It was working fine for most (B2B) orgs before I started doing that, though
    – Aidan
    May 2, 2018 at 8:33

The only locking mechanism I know of is the SOQL for update. (But appears broken at the moment for this scenario - see Webservice Callouts within a Select For Update statement are not blocked per Daniel's comment.)

So I suggest say a custom setting field that is queried/set/unset by the Queueable and when the QueryException results the Queueable re-enqueues itself as this means a callout is already in progress.

  • 3
    Interesting side note on FOR UPDATE - there is a known issue around callouts that drops the lock. May 1, 2018 at 0:41
  • Hey Keith, that's a really interesting approach, have you ran this in production before? It sounds a little risky. I can imagine infinite loops or deadlocks possibly occurring as a result. The issue that @DanielBallinger brought up sounds like it may cause issues
    – Mattisdada
    May 1, 2018 at 6:28
  • @Mattisdada I do not have this in production. Agreed that it looks like its broken at the moment or worse still perhaps never worked. I suppose the only good news is that "known issues" generally do get fixed so keep a watch on this issue.
    – Keith C
    May 1, 2018 at 8:01
  • @DanielBallinger Thanks for that link; more like a blocker at the moment for this approach than a side note though...
    – Keith C
    May 1, 2018 at 8:03

I wrote an article about how you can use some form of in-database tracking state (identifying what needs to be done), Platform Events and a Platform event subscriber to take processing "async" from triggers in a governor friendly manner.

By using Platform Events you actually avoid using daily async limits and instead use from your hourly Platform Event publication limits (which are relatively high). Of course, to do an actual callout you do (sadly) still have to use async.

Note that Apex trigger-based Platform Event subscribers have the characteristic that they are called in a single-threaded manner.

You will find example code attached to the article, which flags the actual records themselves (though does not then go async as needed for a callout) and a second example that uses a separate "command" object to track what must be done (and includes bulkified async for simulated callout support).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .