How to make more than 100 callouts per transaction?

Question

I am new to Salesforce callouts and batch concepts but here is the requirement that I am working on:

Requirement is to get data from a third party API and create lead objects in our salesforce and for this I need to be able to make more than 100 future calls.

For now, I created a simple class and invoke this class from Anonymous Apex window in developer console.

This class makes callouts:

public with sharing class CallOutExample {

public void submitHttpRequest(){
    HttpRequest req = new HttpRequest();

    req.setMethod('GET');

    req.setEndPoint('http://www.example.com/endpoint');

    Http httpObj = new Http();
    HttpResponse res;
    String jsonString;

    List<Lead> leadObjList = new List<Lead>();

    for(Integer j =0; j <105; j++){

        res = httpObj.send(req);
        jsonString = res.getBody();

        Map<String,Object> jsonMapObj = (Map<String, Object>)JSON.deserializeUntyped(jsonString);

        List<Object> leadJsonObjList = (List<Object>)jsonMapObj.get('leadJson');

        for(Integer i=0; i<leadJsonObjList.size(); i++){
            // Here I take values for the last name, first name, phone number and everything from each lead Json object
            // and create lead objects and to the list
            Lead leadObj = new Lead(); //Assuming I am setting all the values
            leadObjList.add(leadObj);
        }   

    }

    Database.insert(leadObjList);

}

}

And then I am invoking this method with the below code from anonymous apex:

CallOutExample obj = new CallOutExample();

obj.submitHttpRequest();

Can somebody point me in right direction on how to achieve this:

If I write a batch class, will I be able to make more than 100 callouts? If so how do I implement the start method of the Batchable interface?

Answer 1

I'm a little confused by the architecture. You're making ~100 callouts, but you have structured your code as if you're getting multiple lead objects in the body of each response. How many leads are you receiving per callout?

So depending on how many leads you get in each response, that will be your batch (scope) size I think.

Something like this:

public with sharing class MyLeadImporterBatchable implements Database.Batchable, Database.AllowsCallouts {

    public static final BATCH_SIZE = 10;

    public Iterable<Lead> start (Database.BatchableContext BC) {

        List<Lead> scope = new List<Lead>();

        for (Integer i = 0; i < BATCH_SIZE; i++) {

            scope.add(new Lead());
        }

        return scope;
    }

    public void execute (Database.BatchableContext BC, List<Lead> scope) {

        // Make callout

        // Parse response, populate scope with contents

        // Insert scope
    }

    public void finish (Database.BatchableContext BC) {

        // Make another callout

        // Parse response
        if (!leadJsonObjList.isEmpty()) {

            Database.executeBatch(new MyLeadImporterBatchable(), 1);
        }
    }
}

So basically:

• start() - creates X Lead records which will be processed during the execute() method
• execute() - makes the callout & parses the response into the lead records, inserts the records
• finish() - makes a second callout to see if there are more records to process and if so, queues up another batch class

I'm not too clear on your architecture but that structure should be pretty close to what you need.