2

Problem

We are launching a Batch Apex process containing 430 batches (200 records each batch). We've ran it 4 times, all times it finished successfully without any error, but not all batches were processed.

In the "Apex Jobs" section you can see:

  • 1° launch: Total Batches: 102 Batches Processed: 102 Failures: 0
  • 2° launch: Total Batches: 105 Batches Processed: 105 Failures: 0
  • 3° launch: Total Batches: 98 Batches Processed: 98 Failures: 0
  • 4° launch: Total Batches: 102 Batches Processed: 102 Failures: 0

The batch process starts showing 430 batches but when it finish shows 105 or less.

enter image description here enter image description here

Question

Could someone tell us what is going on? Why is SFDC changing the initial batch's amount?

Thank you.

Update

This is the apex code:

ReceiptsBatch Methods

global Database.QueryLocator start(Database.BatchableContext BC){
        this.processStatus = true;

        this.process.Status__c = ProcessManagerUtils.STATUS_INPROGRESS;
        update this.process;

        String query = 'SELECT Id,CloseDate,Amount,Bank_Account_Entity__c,Bank_Account_Formula__c,Credit_Card_Type__c,Credit_Card_Number__c,Credit_Card_Expiration_Month__c,Credit_Card_Expiration_Year__c';
        for (String field : ReceiptsBatchHelper.MAP_KEY_FIELDS){
            query += ','+field;
        }
        query += ' FROM Opportunity WHERE StageName = \''+ReceiptsBatch.PENDING_STATUS+'\''; 
        query += ' AND Payment_Method__c in (\'' + ReceiptTriggerHelper.RECEIPT_PAYMENT_METHOD_AUTO_DEBIT 
                    +'\',\'' + ReceiptTriggerHelper.RECEIPT_PAYMENT_METHOD_CREDIT_CARD + '\')';
        query += ' AND CloseDate <= '+String.valueOf(this.process.Receipts_To_Date__c);
        query += ' AND Receipt__c = NULL';
        query += ' AND AccountId != NULL';

        return Database.getQueryLocator(query);
    }

    global void execute(Database.BatchableContext BC, List<sObject> scope){  
        ReceiptsBatchHelper.myProcessId = process.Id;

        List<Opportunity> donations = (List<Opportunity>) scope;
        try{
            ReceiptsBatchHelper.prepareRemittanceBanks();
            ReceiptsBatchHelper.buildDonorAccountIdByOppIdMap(donations);
            this.receiptsByKeys = ReceiptsBatchHelper.prepareDonationsByGroups( donations, 
                                                                                this.receiptsByKeys );
            ReceiptsBatchHelper.upsertReceiptsAndUpdateOpps(this.receiptsByKeys);
        }
        catch (Exception e){
            this.processStatus = false;
            this.errorDescription = e.getMessage() + '. ' + Label.Line_Info + ' ' + e.getLineNumber();
        }
    }

ReceiptsBatchHelper Methods

public static void prepareRemittanceBanks(){
    for (Bank_Configuration__c conf : [SELECT Bank_code__c, Merchant_Bank__c, Merchant_Bank__r.Concept__c FROM Bank_Configuration__c]){
        remittanceBankByBankEntity.put(conf.Bank_code__c, new Remittance_Bank__c(Id = conf.Merchant_Bank__c, Concept__c = conf.Merchant_Bank__r.Concept__c));
    }
}
public static void buildDonorAccountIdByOppIdMap( List<Opportunity> donations ){
    Map<String, Opportunity> oppMap = new Map<String, Opportunity>();
    for( Opportunity opp : donations ){
        oppMap.put(opp.Id, opp);
    }

    for( OpportunityContactRole ocr : [ SELECT Id, OpportunityId, Contact.AccountId, Contact.Id
                                        FROM OpportunityContactRole 
                                        WHERE OpportunityId IN :oppMap.keySet()
                                        AND Role = 'Donor' AND IsPrimary = TRUE] ){
        donorAccountIdByOppId.put(ocr.OpportunityId, ocr.Contact);
    }
}
public static Map<String, Receipt__c> prepareDonationsByGroups( List<Opportunity> donations, 
                                                                Map<String, Receipt__c> generatedReceiptsMap ){
    for (Opportunity o : donations){
        String key = createKey(o);       

        if (generatedReceiptsMap.containsKey(key)){         
            if(groupedOpportunites.containsKey(key)){       
                groupedOpportunites.get(key).add(o);
            }
            else{
                groupedOpportunites.put(key, new List<Opportunity>{o});
            }
            generatedReceiptsMap.get(key).Amount__c += o.Amount;
            receiptsToUpsert.put(key, generatedReceiptsMap.get(key));
        }else{
            Receipt__c newReceipt = generateReceipt(o);
            groupedOpportunites.put(key, new List<Opportunity>{o});
            receiptsToUpsert.put(key, newReceipt);
            generatedReceiptsMap.put(key, newReceipt);
        }
    }

    return generatedReceiptsMap;
}   
public static void upsertReceiptsAndUpdateOpps (Map<String, Receipt__c> receiptsByKeys ){    
    List<Opportunity> opportunitiesToUpdate = new List<Opportunity>();

    upsert receiptsToUpsert.values();

    for (String key : groupedOpportunites.keySet()){  
        for (Opportunity o : groupedOpportunites.get(key)){
            o.Receipt__c = receiptsByKeys.get(key).Id;
            opportunitiesToUpdate.add(o);
        }
    }

    //if there is an error updating the opportunities, we must delete created receipts
    try{
        update opportunitiesToUpdate;
    }
    catch (Exception e){
        delete receiptsToUpsert.values();

        e.setMessage(Label.Error_linking_opportunities_to_receipts + ' ' + e.getMessage());
        throw e;
    }
}
  • 1
    could you please check the number of records that are being processed? – San Tosh Mar 3 '16 at 11:56
  • Yes, the total records processed were (102x200) 20400. The rest of the records until (403x200) 80600 were not processed. – Alberto Sanchez Mar 3 '16 at 12:26
  • 1
    this may be because of CPU timeout issue internally. Try reducing batch size to 50 from 200. The intention being to reduce the resource usage. – San Tosh Mar 3 '16 at 12:30
  • can you show us the code? – Novarg Mar 3 '16 at 13:00
  • 1
    Alberto -- (1) You should be using Database.Stateful to keep track of all errors so they can be emitted during the finish() method; (2) the fact that the change in batch size from 200 to 50 still yielded 20000 rows processed suggests there is some data error/limit error on the rows queried after 20000. To debug this, you may need to change the way the batch start() builds the SOQL by querying from a custom object field or anonymous apex parameter passed to the constructor so you can isolate where this is happening. – cropredy Mar 6 '16 at 8:11
1

Turning the comment thread into an answer

The clue here was when you reduced the batch size from 200 to 50 but still stalled out at record 20000. There could be several reasons for this:

  1. There is something in the data in the batch processed at record 20,000 - 20,050 that causes the entire batch process to stop. Anything from a Limits error (some sort of out-of-control loop) to an Internal Salesforce Error
  2. Or, as you discovered, your Database.Stateful variables exceeded Heap size given that your batch is large (100,000 records). I initially didn't consider this as your code posted didn't include Database.Stateful.
  • The second reason (exceeded Heap size limit) but by other cause. I can say the clue was when I reduced the batch size. I've revised my Stateful variables and one of them had a size over 12 Mb. Because of this, I've done a work around to store less information. Now it's work fine. – Alberto Sanchez Mar 11 '16 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.