2

I have a batch Apex class that makes a callout to an API that returns records in pages of 25. It also tells me how many pages there are so essentially I do - so I have a loop that then goes through those and makes any subsequent calls.

However, if there are over 2500 results... then there are more than 100 callouts being made.

Is there a structure that would help me get around this limit?

    Public static void getAddlLeads(set<string> companyIds) {

       for(string ziCompanyId : companyIds) {
            //Make the call
            Http http = new Http();
            HttpRequest req = new HttpRequest();
            req.setendpoint(fullURL);
            req.setmethod('GET');
            req.setHeader('Accept', 'application/json');
            HttpResponse res = http.send(req);
            String response = res.getBody();

            //return response;
            System.debug('response: ' + response);
            List <ziPersonSearchJSON.PersonRecord> addPeople = new List<ziPersonSearchJSON.PersonRecord>();
            List <Lead> ziAddlLeads = new List < Lead > ();

            ziPersonSearchJSON obj = ziPersonSearchJSON.parse(res.getbody());

            decimal totalResults = obj.PeopleSearchRequest.TotalResults;
            decimal callsReq = totalResults.divide(25,4).round(System.RoundingMode.UP);
            system.debug('need to make calls: ' + callsReq );

            for( ziPersonSearchJSON.PersonRecord person: obj.PeopleSearchRequest.PeopleSearchResults.PersonRecord) {
                addPeople.add(person);
                system.debug('Found: '+person.lastName);
            }

            //make additional calls and put them into the list 
            if(callsReq > 1){
                for( integer i = 2; i <= callsReq; i++ ){
                    //Make the call
                    system.debug('Making PersonSearch: '+i);
                    Http httpA = new Http();
                    HttpRequest reqA = new HttpRequest();
                    reqA.setendpoint(fullURL+'&page='+i);
                    system.debug(fullURL+'&page='+i);
                    reqA.setmethod('GET');
                    reqA.setHeader('Accept', 'application/json');
                    HttpResponse resA = http.send(reqA);
                    system.debug(resA.getbody());

                    try{
                    ziPersonSearchJSON objA = ziPersonSearchJSON.parse(resA.getbody());


 for( ziPersonSearchJSON.PersonRecord personA: objA.PeopleSearchRequest.PeopleSearchResults.PersonRecord) {
                        addPeople.add(personA);
                        system.debug('Found: '+personA.lastName);
                    }
                } catch (Exception e) {
                    //nothing
                    system.debug('nothing');
                }


            }

        }

        for (ziPersonSearchJSON.PersonRecord person: addPeople) {
            lead matchLead = new Lead();
            matchLead.lastName = person.lastName;
            matchLead.firstName = person.firstName;
            matchLead.company = person.CurrentEmployment.Company.CompanyName;
            matchLead.title = person.CurrentEmployment.JobTitle;
            //matchLead.Zoom_Info_Company_ID__c = person.CurrentEmployment.Company.companyId;
            matchLead.website = person.CurrentEmployment.Company.companywebsite;
            matchLead.Zoom_Info_ID__c = person.personId;
            matchLead.email = person.email; 
            matchLead.Zoom_Info_Direct_Line__c = person.phone;
            //matchLead.Zoom_Info_Company_Phone__c = person.CurrentEmployment.Company.CompanyPhone;
            //matchLead.ziLastUpdate__c = date.newInstance(1990, 1, 1);//will trigger ziUpdate
            ziAddlLeads.add(matchLead);
        }

        addpeople = null ;
        obj = null;

        system.debug('found these leads: '+ziAddlLeads);

        //dup checker 
        List<Lead> fl2 = new List<Lead>();

        if(ziAddlLeads.size() > 0) { 
            system.debug('running dup check');
            Set<String> companyOfNewLeads = new Set<String>(); 
            Set<String> emailOfNewLeads = new Set<String>();
            Set<String> fnameOfNewLeads = new Set<String>();
            Set<String> LnameOfNewLeads = new Set<String>();
            for(Lead l : ziAddlLeads)
            {
                if(l.email != null)
                {
                    emailOfNewLeads.add(l.email);
                }
                if(l.firstName != null)
                {
                    fnameOfNewLeads.add(l.firstname);
                }
                if(l.lastname != null)
                {
                    LnameOfNewLeads.add(l.lastname);
                }
                if(l.company != null)
                {
                    companyOfNewLeads.add(l.company.trim());
                }
            }

            Set<ID> lids = new Set<ID>();
            Set<ID> cids = new Set<ID>();
            List <Contact> ctc = [Select id, name, firstname, Zoom_Info_Individual_ID__c, lastname, email, contact.account.Name, focusId__c, focusStatus__c from contact where contact.account.Name in: companyOfNewLeads OR email in: emailOfNewLeads ];
            List <Lead> exisLd = [Select Id, firstName, Zoom_Info_ID__c, LastName, email, company, focusId__c from Lead WHERE (company in: companyOfNewLeads OR email in: emailOfNewLeads) and isConverted = false];

            // Dupe Checker Rules                       
            for(Lead li : ziAddlLeads)
            {
                //names +='The new leadis "' + li.firstname+ '" "' + li.lastname + '" "' + li.company +   '"!!';    
                Boolean found = false;
                for(Lead lp : exisLd)
                {
                    if (( li.firstname == lp.firstname && li.lastname == lp.lastname && li.company == lp.company) || (li.email == lp.email) && li.email != null || li.Zoom_Info_ID__c == lp.Zoom_Info_ID__c )
                    {
                        found = true;
                        system.debug('found '+li.firstName);
                        break;
                    }
                }

                for(Contact c : ctc)
                {
                    if( (li.firstname == c.firstname && li.lastname == c.lastname && li.company == c.account.Name ) || (li.email == c.email) && li.email != null || li.Zoom_Info_ID__c == c.Zoom_Info_Individual_ID__c)
                    {
                        found = true;
                        system.debug('found '+li.firstName);                     
                        break;
                    }                                   
                }
                if(found)
                    continue; 
                if(found == false)
                {
                    fl2.add(li);
                }
            }
            system.debug('new Leads:' + string.valueOf(fl2.size()));

            insert fl2;

        }
    }
}
  • Can we see the Batch apex specifics? – Phil Hawthorn May 24 '16 at 16:33
3

The 100 callouts limit is per execution context, in context to what you are talking about that is each execute() iteration within your batch class.

It sounds to me like you could do to refactor your batch class to so that not all 100 are being consumed in one execute(). In reality, this could mean reducing your batch size down to as little as 1, e.g.

Database.executeBatch(new CalloutClass(), 1);
  • Currently, my batchable looks for records that need to have the call run, and then within the execute is making the calls - so even if my batch size is 1, if there are more than 2500 results returned, it will still hit the limit. Are you suggesting that I move the callout to a new batchable class? Can you call a batchable from a batchable? – Justin May 24 '16 at 15:51
  • 3
    I wasn't suggesting that, I was suggesting you pass the 2500 records back as an iterable from the start method so you could process them 1 by 1 (or, 100 by 100). You can call a batchable from a batchable from the finish() method. You could also use Queueable interface to perform the callouts from the batchable. Would need to see your code really.... – Phil Hawthorn May 24 '16 at 16:00
  • Edited to add code – Justin May 24 '16 at 16:12
  • Would it be possible to have my initial class create an interable object of the URLs that need to be called, and then have another batch run (with a size of 1) for each of those interable objects? How would I invoke that batch call from the current batch call? Also - would I have to use an custom object to hold those URLs? – Justin May 24 '16 at 16:43
  • SInce you're doing one callout per companyId then you could return your Set<Id> companyIds as your iterable in start() then call the WS one at a time – Phil Hawthorn May 24 '16 at 17:00
1

Ok, I was able to do the following...

Scheduled Batch 1 gets the leads that are marked as New and runs the first callout to determine how many results the API has for that company. For each page of results return, it creates a custom object callout__c that has a field for the url.

Scheduled Batch 2 then consumes those callout__c objects making the api call for each and creating the leads returned. At the end of processing them, it then deletes them.

Is this an advisable strategy?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.