Schedule apex class to run every minute as there can only be 5 batch jobs

Question

I have a need to execute the Apex class every minute.

I have a Sites VF page and in that I have link called "Export to Excel" which exports bulk data (say 50000 records of my wrapper class type) through Batch Apex.

The application is used by about 100 users When more than 5 users click on that link simultaneously (at the same time), or within a minute batch jobs fails.

Am planning to have a custom object and make an entry in it whenever the users click on that link and populate whatever necessary details needed for executing that job and will also have a status column. If the status is yet to process and if the number of running process is less than 5 (or) 3, planning to call an apex batch job and this caller class will be a schedulable type and thought of scheduling it every minute. Unfortunately am unable to schedule it for every minute.

Any ideas for doing a custom implementation for scheduling my job every minute without installing any external apps? Or is there an alternative solution for my problem?

Answer 1

You could also create you own custom apex batch queue. I did this myself and created the GitHub project SObject Work Queue which had the following design goals:

• Must prevent Max 5 batch in parallel limit - We should never run into this limit with work that is processed over the queue.
• The queue needs to be so generic that "work" on any type of database object needs to be enqueued.
• Any type of modification of database objects need to be possible. This should be transparently handled by the queue.
• Provides better error diagnostics like Batch. Knows last successful Id, full stacktrace and sends email to developers.
• Secures data integrity like Batch or better. Failures should not leave data in inconsistent state or user of the infrastructure should be able to handle them.
• Optimistic locking : Instead of locking all many records we do not to process work on Ids that have other work already scheduled.
• Work that can be run synchronously, should not be queued and processed asynch.

Maybe you want to check if that could be used in you case or collaborate on GitHub to extend it for your purpose.

Here is an overview sequence diagram that shows how work is defined, enqueued and processed in such a custom Apex Queue:

Answer 2

your problem is cotrolling the maximum number of requests being serviced at a time. In your code which is invoked on the Export Click, you should (as Mohith has pointed out above) query the AsyncApexJob and check how many Batch Apex Jobs are in progress.

You may then depending on how many jobs are running, either

• Display a message to User to try later (not great user experience)

• Add the users requests to a custom object queue, which a scheduled class can then pick up and process, and probably send the extract to the user via email

Another consideration would be to always insert users request into a queue (monitored by a scheduled job ) and have either a time or number of requests threshold, when to pick up all requests and service them in one batch apex run, if volumes are permissible.

Answer 3

datetime mydate = system.now();
        integer d = mydate.day();
        integer m = mydate.month();
        integer h = mydate.hour();
        integer s = mydate.second();
        integer min = mydate.minute();
        integer yr = mydate.year();
        if (min == 59)
        {
            min = 0;
            h = h + 1;
        }
        else min = min + 1;
        SFA_AddressMergeScheduler addmerge = new SFA_AddressMergeScheduler();
        String sch = string.valueof(s) + ' ' + string.valueof(min) + ' ' + string.valueof(h) + ' ' + string.valueof(d) + ' ' + string.valueof(m) + ' ' + '?' + ' ' + string.valueof(yr);
        system.schedule('Address Merge Batch', sch, addmerge);

This will schedule your Job to one minute later .In my case i have job called SFA_AddressMergeScheduler addmerge