API Call out to a Vendor Product from Batch Job [closed]

Question

I need some help in designing the solution.

My requirement is I have around 50,000 contacts in my org. I need to send contacts(external id) to a Vendor-product, get the latest information(eg: email id, phone, address) and update the contact in our Salesforce.com CRM.

Detailed requirement.

1. From Salesforce, take chunk of 10,000 contacts, pass the contact keys(external id) as HTTP REQ Parameter and make a HTTP call to Vendor-Product.

2. Vendor-product accepts this request and creates a batch Job.

3. Make another HTTP call to Vendor-Product to check the status of the Job.

4. Wait for the Vendor-Product job to complete.

5. When Vendor-Product job is complete, make another HTTP call to read the

6. Response (result of the job is in either Jason or CSV format based on my wish).

7. After reading the response (CSV or Jason format), update all 10,000 contacts (email, phonee, address) with latest information from the response.

8. Above process must run for 5 times to completer all 50,000 records.

Please provide some insight on how to implement it. Any code please.

Additional comments Please note that this process won't run daily, may be I would execute this process twice in a year to refresh all my contacts with fresh data(email, phone, address, etc)

DESIGN

• Mainjob –with multiple batch size of 1000 records.

• Mainjob – use Batchable

  • Start
  • Execute
  • Finish
  • Inside Execute() method – Do following
    • Make a HTTP request out to remote service
  • Inside Finish method() – Do following
    • Schedule new Job – JobChild for checking the status and request the result (may be after 10 minutes).

  • Questions:

    • I put a time delay between each batch, so that I can allow remote service to complete the previous request successfully?
    • Can I pass a parameter( that is Remote service JOBID) to a new job –ChildJob ?

  • ChildJob use batchable

    • Make HTTP request to remote service to check the status of the job – Pass JOBID
    • If not complete, reschedule Childjob itself after a time gap.

    • If complete,

      • Make HTTP request to remote service, get the result.
      • Read the response and update the contact records with response.
      • Process the errors.
    • Questions:
      • ChildJob - I don't have any scope. Is that okay to have batachable job without scope ?

Answer 1

Are you required to do 10,000 at a time? If you schedule a batch, you can iterate over all 50k w/o having to run the job multiple times.

In addition, the processing would be sped up - As it is, you may run into several issues, including running times, heap size, and handling any errors for fields you may update.

Generally, as a design, I recommend this:

1. Create a batch job with a size of between 500 and 1000 contacts.
2. Within your controller, schedule a regular job to check back. This job will either reschedule itself, or it will execute a batch job to retrieve the results.

A few things to note:

• Limits are for a single batch in a batch job, so you could make up to 100 callouts per batch
• Depending on how the remote process wants you to get your results, you may want to alter the scheduling so that it waits until all of the batches are complete.

UPDATE: Since you can only process two jobs at a time, and there is no way to force the process to sleep, I recommend allowing the first job to kick off either the wait job or the next job.

This link has two good examples for dealing with parameters to scheduled jobs: Passing Parameter Into Schedulable Class

What you'd want to do for the schedulable job:

global class ContactUpdate_SendBatch implements Schedulable{

    global void execute(SchedulableContext sc) {
        //get next batch of contacts
        List<Contact> contacts = ContactUpdate_BatchingHelper.getNextBatch();
        String jobId = RemoteService.sendBatch(contacts);

        System.schedule('Wait on Batch',nextRunTime, new ContactUpdate_WaitOnBatch(jobId));
    }

    global void finish(Database.BatchableContext BC)
    {
        // Get the ID of the AsyncApexJob representing this batch job from Database.BatchableContext.
        // Query the AsyncApexJob object to retrieve the current job's information.
        AsyncApexJob a = [SELECT Id, Status, NumberOfErrors, JobItemsProcessed, TotalJobItems, CreatedBy.Email 
                          FROM AsyncApexJob WHERE Id = :BC.getJobId()];

        //then use the active job id and abort it
        system.abortJob(a.id);
    }
}


global class ContactUpdate_WaitOnBatch implements Schedulable{
    global String JobId {get;set;}

    global ContactUpdate_WaitOnBatch(String jobIdVal)
    {
        JobId = jobIdVal;
    }

    global void execute(SchedulableContext sc) {
        if(RemoteService.isCompleted(JobId) == false)
        {
            //Reschedule job, since the the finish method removes it
            System.schedule('Wait on Batch',nextRunTime, new ContactUpdate_WaitOnBatch(jobId));
        }
        else
        {
            RemoteService.ProcessBatch(JobId);
            System.schedule('Process Next Batch',nextRunTime, new ContactUpdate_SendBatch(jobId));
        }
    }

    global void finish(Database.BatchableContext BC)
    {
        // Get the ID of the AsyncApexJob representing this batch job from Database.BatchableContext.
        // Query the AsyncApexJob object to retrieve the current job's information.
        AsyncApexJob a = [SELECT Id, Status, NumberOfErrors, JobItemsProcessed, TotalJobItems, CreatedBy.Email 
                          FROM AsyncApexJob WHERE Id = :BC.getJobId()];

        //then use the active job id and abort it
        system.abortJob(a.id);
    }
}

I've glossed over some of the finer details, such as the nextRunTime variable, but you can find this information online quite easily. In addition, I pulled some info from here: https://help.salesforce.com/apex/HTViewSolution?id=000002809&language=en_US

Answer 2

Solutions below.

1-Make a batch class (Batchsize-10,000).

2-Make a HTTP request from web service class.

3-Make a HTTP request for status in while loop with delay 5 minutes.

4-Get the response from status HTTP request, if status done then we make another HTTP request for result.

5-We can save all information in CSV/Jason format after result.

If you want to discuss more about this please email me at [email protected]

Thanks, Sandy