Handling a combination of async callouts, schedulable jobs, and queueable jobs

Question

The current requirements for my system is as follows:

1. When a certain change is saved on the Account object, I need to create an Order object and make a callout to Service A.
2. Service A finishes processing at some point in time, and the only way to tell if it's done is to make a verification check. Once it's verified, I need to make callouts to 6 different services (B, C, D, E, F, G) to notify them of the response.

The current bottleneck is that every single service from A-G are asynchronous, which caused me to use a combination of future callouts, scheduled jobs, and queueable jobs.

The way the system currently works is that after the Order object is created and the callout to Service A is made, I have a scheduled job running every 5 minutes that will check every incomplete Order object and verify with Service A.

Once verification from Service A is received, the Order object is modified, which then fires the Order trigger. The trigger will then need to make 6 callouts, which I put in a Queue.

Previously, it was guaranteed that there would only be one incomplete Order object at a time in the system, but that is no longer going to be true. However, when there are more than one Order object's trigger getting fired, I run into the governor limit of only one queueable job being allowed to exist at a time.

Is there anyway around this? Or will I have to rearchitect the whole thing? Everything is already kind of convoluted because of the need to do DML statements with the callouts, so I'm making insert and update statements in unintuitive places (in the queueable handler). I'm almost considering just spinning up an AWS server or something to handle the 6 asynchronous callouts, but then I would have to deal with security issues and that's more trouble than it's worth right now.

Thank you.

EDIT: Here is the current error message I get: 12:28:04:805 FATAL_ERROR System.LimitException: Too many queueable jobs added to the queue: 2 The stack trace points that this error happens on the line where I run 'System.enqueueJob(new PbxQueueableJobs());The line that caused the error is in a trigger that is fired due to a mass update of twoOrderobjects. The error was caused by the secondOrder` object's trigger.

Answer 1

The limit you're seeing is that in async transaction context you can only add one job to the queue with System.enqueueJob. Queueable jobs can be chained and chaining works regardless of whether a job does or does not do a callout.

From the doc:

To run a job after some other processing is done first by another job, you can chain queueable jobs. To chain a job to another job, submit the second job from the execute() method of your queueable class. You can add only one job from an executing job, which means that only one child job can exist for each parent job. For example, if you have a second class called SecondJob that implements the Queueable interface, you can add this class to the queue in the execute() method as follows:

public class AsyncExecutionExample implements Queueable {
      public void execute(QueueableContext context) { 
        // Chain this job to next job by submitting the next job
        System.enqueueJob(new SecondJob());
      } 
}

If you implement one Queueable job per target (B-G), you should be able to fire them in a chain and stay under one queued job per async transaction limit.

You should also take a look at Integration Patterns for possible rearchitecture ideas.