Batch apex scheduling

Question

I am calling multiple Batch Apex classes (more than 5) from another apex class and I want to build apex logic so that it doesn't kick off next batch apex if there are already 5 running/queued Apex Jobs. It should kick off the next batch apex as soon as one of the 5 active Apex batch finishes. And this cycle should continue till the total number of apex jobs are complete. I want to build efficient logic avoiding any infinite/unnecessary loops.

Answer 1

What you will have to do is check the number of running jobs, I have built my own pattern to do this and here is an excerpt of code from that:

private void submitJob( IReschedulable job, SchedulableContext SC )
{
    if( getCurrentJobCount() > 4 )
    {
        // try again in a minute
        Datetime sysTime = System.now().addSeconds( 60 );

        String chronExpression = '' + sysTime.second() + ' ' + sysTime.minute() + ' ' + sysTime.hour() + ' ' + sysTime.day() + ' ' + sysTime.month() + ' ? ' + sysTime.year();

        System.schedule( job.getJobName() + sysTime, chronExpression, (System.Schedulable)job );
    }
    else
    {
        Database.executeBatch( (Database.Batchable<Object>)job, job.getBatchSize() );

        // abort scheduled job if this was as a result of a reschedule
        if( SC != null && job.getAbortAfterSubmit() )
        {
            System.abortJob( SC.getTriggerId() );
        }
    }
}

What I do is check the number of batch jobs running, and if its greater there are no resources to process it, schedule it to run again in a minute. If there are resources, obviously run it. This method of re-scheduling has been given the unfortunate moniker of "Suicidal Apex" (by some in Salesforce because it essentially 'kills' itself after running) and comes with a health warning. That said, it works perfectly for me and is prolific in my code base.

You mention "It should kick off the next batch apex as soon as one of the 5 active Apex batch finishes" - for this, I believe you will have to implement some kind of queue mechanism that does some kind of polling of the AsyncApexJob table but its difficult to see how you could avoid infinite/unnecessary loops unless you were somehow able to utilse the Streaming API.

Be interested to know what solution you come up with, good luck.

Answer 2

I would recommend checking out Dan Appleman's sessions at DF13 where he discusses some patterns you can use for continuous async processing - you can view the session here

I watched it this week and it has some nice solutions to problems like this