1

We have a requirement of operating on data in present in different objects. The total record count (of all objects) ~ 50 millions (some objects holding ~20 millions records alone). So we have considered the option of handling it via batch apex.

The logic is:

  • Create a List (say objectsList) of String which will hold the object names.
  • A method will start batch execution from For loop. Batch apex will identify the object by the integer passed and start operating on that object.

Something as follows:

for(integer i = 0; i< objectList.size(); i++){
RandomBatch bch = new RandomBatch(i);
Database.executeBatch(bch,2000);
}

Is this feasible and practical? From what I read, flex queue has capacity of holding 100 jobs in queued state. We don't have more than 20 objects to cover (so 20 batches will be executed). Can flex queue hold jobs for over 2 days (since objects holding ~20 millions records are going to take time to complete). And is it even acceptable to execute batch apex from For loop? There's nothing going to be in finish() method (apart from mailing us the error messages if any).

6
  • What is that batch doing?, just dml updates? Nov 16, 2019 at 13:45
  • Yes - Updating set of fields for every object. Nov 16, 2019 at 13:46
  • why not data-loader, and use bulk api? Nov 16, 2019 at 13:47
  • Because the set of fields (defined for every object) keeps changing and we are maintaining the list of fields using Custom Setting. We need to setup a process for such dynamic requirement Nov 16, 2019 at 13:49
  • So our planned batch class will simply pick the list of fields to be updated from custom setting and start update operation Nov 16, 2019 at 13:50

1 Answer 1

3

Personally, I have not seen Batches to be queued that long, even if they are it doesn't seem SF will kill them after some time if they are in limit.

That being said, You can follow another approach. We know at max 5 batch jobs can run at a time. Let's utilize this and start the next batch at the end of each batch.

RandomBatch bch = new RandomBatch(0);
Database.executeBatch(bch,2000);
bch = new RandomBatch(1);
Database.executeBatch(bch,2000);
bch = new RandomBatch(2);
Database.executeBatch(bch,2000);
bch = new RandomBatch(3);
Database.executeBatch(bch,2000);
bch = new RandomBatch(4);
Database.executeBatch(bch,2000);

And in the finishMethod of each batch, you will add 5 to your Index and run next batch till the index reaches the max value of 20.

public void finish(BatchableContext bc){
   Integer nextBatchIndex = currentIndex + 5;
   if(nextBatch < 20){
      RandomBatch bch = new RandomBatch(nextBatchIndex );
      Database.executeBatch(bch,2000);
   }
}

Also you need to ensure that your batch won't exceed the Async Apex Limit , because if that happens you wont be able to run any future queuable schduler apex till the limit resets.

1
  • Thanks for this Pranay - As mentioned, the object list size will keeps changing (won't remain 20). But can definitely take this approach of chaining. Nov 16, 2019 at 14:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .