Processing batches with a delay

Question

I have an interesting challenge, I'm sure there are a few decent solutions but I'd like to get some input from the community. I am building a tool that will allow my users to use the Angel IVR Outbound API (you call it API, it places calls for you. Like Twilio). They run a custom query using the tool and it call's all the resulting records. I have this part working just fine. The challenge is, we want to break the calls up into delayed batches. Reason being that if we place 1000 calls and they all go out immediately and then the people receiving the calls need to call into our customer care center, our help desk people could be quickly overwhelmed by the call volume. So ideally we could pass in a set of IDs and a minute delay as well and have the batch of 1000 broken into groups of say 200 with that minute delay inserted between.

The approach I was thinking of taking is having the main application look and see if a delay time greater than 0 is passed in. If so, instead of just batching and placing the calls (my outbound dialing function already handles the batching) it would schedule an invocation of a scheduable class. That class would then take the entire list of ID's (would have to store them somewhere, not sure where though since I don't think you can pass arguments to a scheduled class). That scheduled class would send one batch and delete those ID's from the set. If there are ID's left it will schedule another invocation of itself X amount of time from now, which will then recursively run until all ID's have been handeled. This seems solid, but there are a few questions.

1) Can a scheduled class schedule another invocation of itself? 2) What would be the best way to store the list of records to dial

Just to get an idea of scope, about the most records we would every deal with this application would be about 10,000, but ideally it could scale up near infinity. The batch size will generally be between 100 and 250. Each batch consists of just one outbound HTTP request to the angel API, as you can include multiple numbers to call per request (up to 250, hence the batch size limit).

Anyway, I'd love to hear some input from other experts and see what you all think might be the best way to approach this. If you are curious what I have for my class already check out my blog at

http://iwritecrappycode.wordpress.com/

Particularly the campaignCall function, as that is the one that gets called currently to place calls. Thanks!

Answer 1

The strategy that I have seen employed to do this is to schedule a job one minute in the future, passing the state between the Scheduled and Batch Apex class as a property so you have some scope. So, in the finish method of your Batch Apex class, implement a 'delay':

Datetime sysTime = System.now();
sysTime = sysTime.addSeconds(60);
String chron_exp = '' + sysTime.second() + ' ' + sysTime.minute() + ' ' + sysTime.hour() + ' ' + sysTime.day() + ' ' + sysTime.month() + ' ? ' + sysTime.year();
ScheduledClass scheduler = new ScheduledClass();
scheduler.SomeState = yourState; 
//Schedule the next job, and give it the system time so name is unique
System.schedule('CMP2 ' + sysTime,chron_exp,scheduler);

Schedulable class:

global with sharing class ScheduledClass implements Schedulable
{
    public Set<ID> SomeState {get;set;}

    global void execute(SchedulableContext sc) 
    {  
        BatchClassImpl batch = new BatchClassImpl(doNextBatch); 
        batch.SomeState = SomeState;
        Database.executeBatch(batch,250);
       //remove from scheduled job list
        System.abortJob( sc.getTriggerID() );
    }
}

Answer 2

In Summer '13 release you will have a new "ScheduleBatch" method to use. This will let you specify a time delay for enqueuing the next batch job. It more-or-less does the mechanics that are listed in Phil Hawthorn's response.

Answer 3

The way I look at it, you would need a 'Staging' Custom Object, into which you insert the results of your query as records, with a status (Pending / In Progress / Completed)

This object could have a lookup to the CampaignMember / Contact / Lead whose details need to be sent out via the API Call.

Write a Scheduled Class, which scans this staging table and picks up records with a Status of 'PENDING' with a limit of the number of records you want to process at a go (batch size)

Process these selected items and mark them Completed or delete them.

In this way, the scheduled job runs as often you desire (using System.schedule and a cron expression) and picks up batch size number of records at a go, and processes them.

Answer 4

If I properly understand the problem then this may work:

Persist all the IDs into a custom object with a status as described above. Then you can schedule an apex class to execute every 5 minutes (You can schedule apex to a minimum of every 5 minutes).

This would mean every 5 minutes your table is scanned for queued records, calls the API, then deleted or updates status of the records.

This would allow you to process batches every 5 minutes- hopefully that is suitable.