Just assume you had an app that would have to process huge amounts of records in an org in multiple ways each day. For each "way" you have a separate Batchable class and those are triggered independently by user actions. Depending on how many users and how much data you have you can easily run into one of the Batch related limits.
My ideas was to in any way centralize batch processing and by that reducing the risk of e.g. too many batches running in parallel.
I thought of having a ContinuousWorkerBatch class that checks in his start() method if work needs to be done (which type of work on which record). If so it passes the Ids of the records to process to his execute method otherwise skips the execute. In his finish method it would calls a new instance of itself. With Summer '13 we can use the new method to delay the new batch by scheduling it in a minute.
global class ContinuousWorkerBatch implements Database.Batchable<SObject>, Database.Stateful {
private BatchQueueItem__c currentWork;
private List<Id> recordIds;
private BatchWorkerCommand command;
public ContinuousWorkerBatch() {
this.currentWork = [SELECT txt_CommandClassName__c, ltxt_IdsToProcess__c
FROM BatchQueueItem__c
ORDER BY CreatedDate ASC
LIMIT 1];
this.command = newFromName(this.currentWork.txt_CommandClassName__c);
this.recordIds = parseToList(this.currentWork.ltxt_IdsToProcess__c);
}
global List<SObject> start(Database.BatchableContext context) {
return queryRecordsForIds(this.recordIds);
}
global void execute(Database.BatchableContext context, List<SObject> scope) {
this.command.execute(scope);
updateRecordIds(scope);
}
global void finish(Database.BatchableContext context) {
ContinuousWorkerBatch same = new ContinuousWorkerBatch();
if(this.recordIds.isEmpty()) {
delete this.currentWork;
Database.scheduleBatch(same, 1);
}
else {
this.currentWork.ltxt_IdsToProcess__c = String.join(this.recordIds, ',');
update this.currentWork;
Database.executeBatch(same);
}
}
...
}
But the basic idea is that only a single batch is constantly running and not many are independently triggered and fighting against each other.
Work could be registered by adding record Ids and the name of a Command pattern like class in the database. Maybe just a long text field to be able to store thousands of pairs.
public interface BatchWorkerCommand {
void execute(List<SObject> recordsToBeProcessed);
}
Order of execution could be respected and we would avoid race conditions between batches.
- Is this a stupid idea after a few glasses of wine ;) or a feasible solution?
- Have you done something similar and can share some lessons learned?
- What what you change improve or take into account?
Any feedback is appreciated.
