Take the 2-minute tour ×
Salesforce Stack Exchange is a question and answer site for Salesforce administrators, implementation experts, developers and anybody in-between. It's 100% free, no registration required.

I have a read query in a batch class which in orgs with a lot of data can bring back over 50,000 records and violate the governor limit. My question is it possible to do a count query, realise that the records is over 50,000 and then split the query up into a numer of separate queries to lower the probability of the governor limit being violated?

share|improve this question
    
Any reason why you can't run your batch multiple times on the next batch of 50K objects ? –  Samuel De Rycke Jan 30 '13 at 15:45
    
If I can set the batch size dynamically. can I? –  dublintech Jan 30 '13 at 15:51
    
Are you making this query within the execute method, or are you talking about the queryLocator used in the start method? See my answer: the queryLocator used in the start method can return up to 50 million records, and limits are reset each time that the execute method is called. –  zachelrath Jan 30 '13 at 17:58
add comment

3 Answers 3

up vote 4 down vote accepted

Regarding "If I can set the batch size dynamically. can I?" comment:

You can decide upfront. Use optional param of the method that kicks off the batch. Database.executeBatch(new MyBatchClass(), 10); will walk through "whatever comes from query locator" in chunks of 10 records at a time.

Again - planning upfront - can you maybe use a helper object with a COUNT rollup summary of the stuff you want to iterate over? This needs a master-detail between the two though so might be not working for your case.

You can also make a "poor man's counter" yourself, keeping count of records in a custom setting (updating it with after insert triggers). Or read about Analytic Snapshots.

If we could see the query or know a bit more about your context a bit more, you might get better advice. Fastest way to chunk the data is to use this extra parameter as it requires no further setup & logic.

share|improve this answer
add comment

You don't need to do any sort of count query or roll-up summary at all --- a single Batch Apex class could iterate over millions of records if it needed to.

All of the answers and comments so far seem to be ignoring / forgetting some fundamental characteristics of Batch Apex and its Governor Limits:

  1. The 50,000 records-returned-by-SOQL Limit is for a single transaction -- but each time that the your Batch Apex class's execute() method is called constitutes a new transaction, so that Limit is reset every time that method is called. This means that, if you needed to, you could query for up to 50,000 records each time that the execute method is called! So, perhaps you need to run some logic on all Contacts in your org, but do it based on the parent Account. Well, then in the start method, grab all Accounts, but then in the execute method, query for all of the Account in scope's Contacts (up to 50,000), and update them appropriately!
  2. The initial query locator that is defined in your start() method can return up to 50,000,000 (yes, 50 million) records. So, @dublintech, if your logic just needs to apply to all records returned that match your query, you are totally fine! Say you wanted to mass change the Owner of all 10 million Leads in your org. The Batch Apex is simple:

    public class MassUpdateOwner implements Database.Batchable<sObject>, Database.Stateful {
    
        public Id targetUserId;
    
        public MassUpdateOwner(String ownerUsername) {
            User u = [select Id from User where Username = :ownerUsername limit 1];
            targetUserId = u.Id;
        }
    
        public Database.QueryLocator start(Database.BatchableContext ctx) {
            return Database.getQueryLocator([select OwnerId from Lead]);
        }
        public void execute(Database.BatchableContext ctx, List<SObject> scope) {
           List<Lead> leads = (List<Lead>) scope;
           for (Lead l : leads) l.OwnerId = targetUserId;
        }
        public void finish(Database.BatchableContext ctx) {
            // Send an email or something
        }
    }
    

And this can be called like this:

 MassUpdateOwner b = new MassUpdateOwner('john@acme.com');
 Database.executeBatch(b);

For scenario 1, where you want to loop over all Contact records for each Account, you want to explicitly limit the scope of each batch to be 1, so that Batch Apex will query for all Accounts that you want, but loop over them 1 at a time, so that you can do your query on each Account's contacts. To force the batch size to be 1, just pass in the batch size as an additional parameter:

 Database.executeBatch(b,1);
share|improve this answer
add comment

No. It is not possible. If the logic is going to bring by 50,001 records it doesn't matter if you split it up into two queries one being 49,999 and the other bringing back 2. You still break the limit.

share|improve this answer
    
It all depends on how you architect your Batch class. With Batch Apex you can iterate over 50 million rows (individually if needed! -- e.g. a separate call to the execute() method with only 1 row being passed in) and then within each execute method call make additional SOQL queries as needed, returning up to 50k records. So at the end of you Batch Job, you may have actually queried for and touched some 50,000 * 50,000,000 records. –  zachelrath Jan 30 '13 at 18:01
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.