4

Is it possible to use a iterable batch job to page through a web services result set?

I need to import a potentially large number of records into Salesforce via a third party web service. They currently have a method that returns all the possible records, but it runs into callout timeout and SOQL limits when the number of records exceeds a certain threshold.

Instead, I'd like to use a batch apex job to process a finite number of records from the web service at a time. I'm proposing that the web service be extended with two new methods. One that returns the total number of records to be processed and another that returns a subset of records within a range (basically paging support). A iterable batch job could then be created to pull back each page of results and process them within the timeout and SOQL limits.

My concern is the following warning in the Using Batch Apex docs:

If you use an iterable, the governor limit for the total number of records retrieved by SOQL queries is still enforced.

Is this limit only applied within each batch execution or does it apply for the entire batch job?

I suspect the former, but I'd like some confirmation before heading down this path.

In the same doc page there is also:

Each execution of a batch Apex job is considered a discrete transaction.

Which to me implies the governor limits are per execution as well rather than across the entire batch job.

Improve this question
1
  • I think for each batch execute iteration the governor limit shall apply .So after every iteration at start we will have full 50000 query rows
    – Mohith Shrivastava
    Commented Jan 18, 2013 at 2:54

1 Answer 1

Reset to default
2

If you use an iterable, the governor limit for the total number of records retrieved by SOQL queries is still enforced.

The docs here are talking specifically about the start method.

The distinction being made is that if you return a query locator from your start method, then the normal query governor is bypassed, whereas if you return an iterable, any SOQL executed from within the start method is subject to the normal governors - it is presumed that you will query the database to populate your iterator.

"Each execution of a batch Apex job is considered a discrete transaction." Which to me implies the governor limits are per execution as well rather than across the entire batch job.

Yes, each batch gets its own governor scope.

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .