50,000 query row governor limit

Question

I have a read query in a batch class which in orgs with a lot of data can bring back over 50,000 records and violate the governor limit. My question is it possible to do a count query, realise that the records is over 50,000 and then split the query up into a numer of separate queries to lower the probability of the governor limit being violated?

Answer 1

Regarding "If I can set the batch size dynamically. can I?" comment:

You can decide upfront. Use optional param of the method that kicks off the batch. Database.executeBatch(new MyBatchClass(), 10); will walk through "whatever comes from query locator" in chunks of 10 records at a time.

Again - planning upfront - can you maybe use a helper object with a COUNT rollup summary of the stuff you want to iterate over? This needs a master-detail between the two though so might be not working for your case.

You can also make a "poor man's counter" yourself, keeping count of records in a custom setting (updating it with after insert triggers). Or read about Analytic Snapshots.

If we could see the query or know a bit more about your context a bit more, you might get better advice. Fastest way to chunk the data is to use this extra parameter as it requires no further setup & logic.

Answer 2

You don't need to do any sort of count query or roll-up summary at all --- a single Batch Apex class could iterate over millions of records if it needed to.

All of the answers and comments so far seem to be ignoring / forgetting some fundamental characteristics of Batch Apex and its Governor Limits:

1. The 50,000 records-returned-by-SOQL Limit is for a single transaction -- but each time that the your Batch Apex class's execute() method is called constitutes a new transaction, so that Limit is reset every time that method is called. This means that, if you needed to, you could query for up to 50,000 records each time that the execute method is called! So, perhaps you need to run some logic on all Contacts in your org, but do it based on the parent Account. Well, then in the start method, grab all Accounts, but then in the execute method, query for all of the Account in scope's Contacts (up to 50,000), and update them appropriately!

2. The initial query locator that is defined in your start() method can return up to 50,000,000 (yes, 50 million) records. So, @dublintech, if your logic just needs to apply to all records returned that match your query, you are totally fine! Say you wanted to mass change the Owner of all 10 million Leads in your org. The Batch Apex is simple:

   public class MassUpdateOwner implements Database.Batchable<sObject>, Database.Stateful {
   
       public Id targetUserId;
   
       public MassUpdateOwner(String ownerUsername) {
           User u = [select Id from User where Username = :ownerUsername limit 1];
           targetUserId = u.Id;
       }
   
       public Database.QueryLocator start(Database.BatchableContext ctx) {
           return Database.getQueryLocator([select OwnerId from Lead]);
       }
       public void execute(Database.BatchableContext ctx, List<SObject> scope) {
          List<Lead> leads = (List<Lead>) scope;
          for (Lead l : leads) l.OwnerId = targetUserId;
       }
       public void finish(Database.BatchableContext ctx) {
           // Send an email or something
       }
   }
   

And this can be called like this:

 MassUpdateOwner b = new MassUpdateOwner('john@acme.com');
 Database.executeBatch(b);

For scenario 1, where you want to loop over all Contact records for each Account, you want to explicitly limit the scope of each batch to be 1, so that Batch Apex will query for all Accounts that you want, but loop over them 1 at a time, so that you can do your query on each Account's contacts. To force the batch size to be 1, just pass in the batch size as an additional parameter:

 Database.executeBatch(b,1);

Answer 3

No. It is not possible. If the logic is going to bring by 50,001 records it doesn't matter if you split it up into two queries one being 49,999 and the other bringing back 2. You still break the limit.