First error: Exceeded maximum time allotted for callout (120000 ms)

Question

I have a Batch class making an API callout and I have scheduled this class. I have used a for loop to iterate through the page number variable in my API endpoint. There are 64 pages. Strangely, if I use for(integer i=1; i<=64;i++) I get an error, but it works perfectly fine with for(integer i=1; i<=35;i++) or any lower value than 35 for i. I have tried changing my batch size but doesn't help.

These are my classes:

Batch Class:

  global class ImportReports implements Database.Batchable<Reports__c>, Database.AllowsCallouts 
  {

      global static HTTPResponse getReports(String api)
      {


        HttpRequest req = new HttpRequest();      
        req.setEndpoint(api);
        req.setMethod('GET');
        req.setTimeout(120000);

        Http http = new Http();
        HTTPResponse res = http.send(req);


         return res;      


      }




  global List<Reports__c> start(Database.BatchableContext BC)
  { 

    List<Reports__c> reportsToAdd = new List<Reports__c>();


    for(integer i=1; i<=32;i++)
    {

        String allReportsString = getReports('https://api.example.com/v1/reports/page[number]='+i).getBody();

        WrapperClass allReports = WrapperClass.parse(recentReports);



        for(WrapperClass reportInArray : allReports)
        {
            Reports__c rep = new Reports__c(); 

            rep.id__c = reportInArray.id;


            reportsToAdd.add(rep);
            System.debug(reportInArray.id);
        }



    }


   return reportsToAdd; 


  }    


   global void execute(Database.BatchableContext BC, List<Reports__c> scope)
   {

        try 
      {
            upsert scope;
      } 

      catch (DmlException e) 
      {
              // Process exception here
         }

    }     

    global void finish(Database.BatchableContext BC)
    {     
    } 
}

Scheduling Class:

global class SchedulerClass implements Schedulable, Database.AllowsCallouts{

    public static String sched = '0 11 11 * * ?'; 


        global static String scheduleBatch() {
        SchedulerClass SC = new SchedulerClass(); 
        return System.schedule('Insert Reports', sched, SC);
    }    


    global void execute(SchedulableContext sc) 
    {

        ImportReports b = new ImportReports ();
        ID batchprocessid = Database.executeBatch(b,50);   




    }

  }

How can I resolve this error?

Answer 1

The callout timeout is tricky, because it applies both to single callouts and as a cumulative limit across a transaction:

The maximum cumulative timeout for callouts by a single Apex transaction is 120 seconds. This time is additive across all callouts invoked by the Apex transaction.

Here, it looks like you're running into the cumulative version of this limit. You can send up to ~35 (the number probably varies slightly depending on network speed and server load) callouts in a single transaction because each one takes a little under 4 seconds, on average, to complete.

You're currently doing all of this work in your start() method. You need to make some architectural changes to split this work across multiple transactions in order to complete it successfully. start() runs in a single transaction, and is intended to return a query or iterator that defines the objects which will be split into separate batches, each of which is processed via execute() in a separate transaction.

Inasmuch as you're not iterating over batches of sObjects, I would actually suggest making this into a Queueable chain rather than a batch class. Basically, what you'd do is make a Queueable Apex class like this:

public class MyQueueable implements Queueable {
    Integer startNumber = 0;
    Integer endNumber = 0;

    public MyQueueable(Integer start, Integer end) {
        startNumber = start;
        endNumber = end;
    }

    public void execute(QueueableContext context) {
        // make your callouts here - 
        // process, say, 30, then chain into a new Queueable:
        for (Integer i = startNumber; i < endNumber && i < startNumber + 30; i++) {
            // Do callout and store result
        }

        insert results; // however we accumulated them.
        // Check if we have more work to do and recurse.
        if (i < endNumber) {
            System.enqueueJob(new MyQueueable(startNumber + i, endNumber));
        }
    }
}

This is a very rough sketch, but I think the pattern will be more applicable to your challenge than Batch Apex as you're trying to use it.