0

I need some architectural advice for our managed application.

Our managed app runs a job every 5 minutes and as part of this job we need to run multiple queries. The number of queries could be in the range 100-200. Now, salesforce has a limitation on the number of queries that can be issued per transaction. So we figured that we would need to break these queries into multiple batch jobs and have around 50 or so queries running per batch.

However, we also have requirements in terms of the execution order of these queries. We need to run these queries in a specific order and only process the left over queries in follow up batch jobs. So, if we have around 170 queries, lets call them Q1->Q170

we would process Batch 1 -> Q1-Q50 queries

Batch 2 -> Q51 - Q100 queries

Batch 3 -> Q101- Q150 queries

Batch 4 -> Q151 - Q170 queries

and then start over, so Batch 5 -> Q1-Q50 queries in the same order

I presume we would have to maintain some state per batch job and execute based on that. Any advice on how we can achieve this? Is there any design pattern to achieve this where we can run async jobs in a synchronous manner?

1

While most batchables return records from the start method via this signature:

public Database.QueryLocator start(Database.BatchableContext jobId)

this signature is also supported:

public System.Iterable start(Database.BatchableContext jobId)

and as any list (or array) implements System.Iterable you can return pretty much a list of whatever you want and still leverage separate governor limits per execute, the batch size flexibility and the guaranteed order.

A simple example:

public class MyBatchable implements Database.Batchable<String> {

    public String[] start(Database.BatchableContext bc) {
        ...
    }

    public void execute(Database.BatchableContext bc, String[] scope) {
        ...
    }

    public void finish(Database.BatchableContext bc) {
        ...
    }
}

So if each of your queries only returns a modest number of records, you can use this mechanism to divide them up into groups of say 50 without needing any additional state.

0

you can try using a queueable interface - now you can only queue two jobs at once using the queueable interface - so if you want to queue these to run synchronously - i would do something like this...

  1. Queue the initial batch ( likely from scheduler ) - passing along something to chain to the subsequent batch job in the constructor
  2. in the batch finish - queue the next group - passing along the data that has been managed so far and repeating this until we have reached the end of this chain.

Example code might look like ...

  public class SomeQueueController implements Queueable{
        Object myPassalongObjectHere { get; set; }

        public SomeQueueController( object [ some variable ] ){
            .....
            do any contructor operations here 
        }
  }

  public void execute(QueueableContext context ){
       MyBatchClass bc = new MyBatchClass(mypassAlongObjectHere);
       Database.executeBatch(bc, 10);
  }

}


global class MyBatchClass implements Database.Batchable{
    Object my passalongObjectHere;

    global MyBatchClass(Object myParam){
       ....
    }

    global Iterable<SObject> start(Database.BatchableContext bc ){

    }

    global void execute( BatchableContext bc, List<SObject> scope_data){
       ....
    }

    global void finish(BatchableContext bc){
         SomeQueueController qc = new SomeQueueController(myPassAlongObjectHere);
         System.enqueueJob(qc);
    }
}

using like this you can get around the chaining limits and these will execute in order you set up by whatever you are using to track ( myPassAlongObject ) within your wrapper / record set - typically i use a map to pass along between so i can pull a specific key to determine my order if they are dependent executions - and handle removing the indexes i have already covered within the finish method as well - often i will pass any number of things along between these - Lists, sets, maps, sObjects - anything you need the next group to see or use that can be passed along to give the "Stateful" feel to your process.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.