I have a long running asynchronous process that process millons of records in a CustomObject__c. To prevent other limits we have split this into 2 Batch classes that work on this table in parallel. So whenever a user starts this process in the Apex Logs he will see two Batches starting at the same time.
This perfectly works in the Sandbox but in one customer org after processing a few hundred jobs one of the Jobs is aborted. Support says it is because a query executed in the job took longer than 120 sec.
Question 1: Why is that? The query is data- and selectivity-wise the same for ALL jobs.
Question 2: Why does the Total # batches immediatelly changes on abortion? Is It allways start with 22.000 jobs and after abortion it only shows 12.00.
Sandboxes and Production org are not really the same. On Production servers the optimzation processes that look for problematic jobs are much more pessimistic. So something that will be tolerates in a Sandbox fails in Production.
As both Batches read from the same input table at a certain point one Batch locks out the other so this waits for the query to come back for more than 120 sec.