Apex Trigger Skipping Its Duties Under Load - Without Limits Errors

Question

I've got a code problem that is only surfacing under load but isn't producing limits errors, making it a little tough to figure out what I might need to do to fix it.

==

I've written an insert/update trigger handler method on Contact to post-process daily import data.

It works great in the sandbox I wrote it in (whether running its test method, upserting a little data, or upserting a lot of data).

Deployed to production, the test method passes and it behaves as expected when upserting just a little bit of data (e.g. 2 records).

But as soon as I upsert, say, 100 records, it doesn't do anything at all. Looking through the log, the "Method Entry" & "Method Exit" lines are adjacent, without even a trace of a "statement execute" of its first line (an "if" deciding whether to proceed).

==

Background: When I first upsert the daily import data (from CSV, via Jitterbit), 95% of the data gets put into "holding fields" that I have deemed "overwriteable." So, for example, "JitterbitFirstName" instead of writing straight to "FirstName."

Two fields that come in with every upsert in this process are "OtherDatabaseUniqueID" & "JitterbitUpsertTimestamp."

My goal is to post-process every update/insert of "Contact," figure out if it's due to one of these Jitterbit-upserts, and if so, "smart-copy" data into the "normal" fields. (e.g. "If FirstName is blank, copy JitterbitFirstName into FirstName.")

==

The existing trigger was a "Before" trigger, so I wrote my handler accordingly (as far as I can tell, it wasn't really going to matter which type of trigger I used).

Any ideas why our production org seems to be simply "opting out" of running my helper method at higher volumes of data?

(I can see it "not bother" in the logs by anonymously executing UPDATE [Select ID from Contact WHERE DailyDump_Contact_Id__c <> null LIMIT 1];, LIMIT 30, LIMIT 80, etc. 30 still enters the method & does VARIABLE_SCOPE_BEGIN...newContacts|List<Contact>... and loops through the 30 records after METHOD_ENTRY. 80 just shows adjacent METHOD_ENTRY & METHOD_EXIT lines - but no complaints about reaching limits, either.

Similarly, when actually upserting 2 records, the data copies just fine. When upserting over 100, the upsert occurs - for example, I can see the timestamp change with an update - but none of the before-trigger "post-processing" occurs.

Also, other methods getting called by the trigger, both before & after mine, show evidence in the logs that they're actually getting entered, even at 80 records.)

Here's the trigger:

trigger ContactTrigger on Contact (after undelete, before delete, before update, before insert, after insert) {
    if (trigger.isBefore && (trigger.isUpdate || trigger.isInsert)) {
        // Copy daily-dump data over from "holding fields" to "main fields," if applicable
        ContactTriggerHandler.copyDailyDumpDataBeforeInsertOrUpdate(Trigger.new, Trigger.oldMap);
    }
    ...
}

And here's the helper method:

public with sharing class ContactTriggerHandler {
...
    public static void copyDailyDumpDataBeforeInsertOrUpdate(List<Contact> newContacts, Map<Id, Contact> oldContacts) {
        // Make sure this method only runs once.
        if (alreadyRanDumpCopying==TRUE) {return;}
        alreadyRanDumpCopying = TRUE;

        for (Contact c : newContacts) {
            // Process the "DailyDump" new records
            if (c.DailyDump_Contact_Id__c <> null &&
                c.DDmp_Jitterbit_Most_Recent_Load_DateTime__c <> null &&
                (oldContacts == null ||
                 oldContacts.get(c.Id).DDmp_Jitterbit_Most_Recent_Load_DateTime__c == null ||
                 oldContacts.get(c.Id).DDmp_Jitterbit_Most_Recent_Load_DateTime__c < c.DDmp_Jitterbit_Most_Recent_Load_DateTime__c)
               ) {
                   if (c.ThirdDBID__c==null && c.ThirdDBID__c<>null) c.ThirdDBID__c = c.ThirdDBID__c;
                   if (c.Birthdate==null && c.DDmp_Birthdate__c<>null) c.Birthdate = c.DDmp_Birthdate__c;
                   if (c.Company_Holding_Spot__c==null && c.DDmp_Account_Name__c<>null) c.Company_Holding_Spot__c = c.DDmp_Account_Name__c;
                   if (c.Email==null && c.DDmp_Email__c<>null) c.Email = c.DDmp_Email__c;
                   if ((c.FirstName==null || Test.IsRunningTest()) && c.DDmp_FirstName__c<>null) c.FirstName = c.DDmp_FirstName__c; // To make debug logs easier to read, our test to make sure this works starts with a FirstName in place, unlike how we actually want things to behave, hence the alternative check for isRunningTest for overwriting.
                   if (c.Gender__c==null && c.DDmp_Gender__c<>null) c.Gender__c = c.DDmp_Gender__c;
                   if (c.MailingStreet==null && c.DDmp_MailingStreetLine1__c<>null) c.MailingStreet = c.DDmp_MailingStreetLine1__c +
                       ((c.DDmp_MailingStreetLine2__c <> null) ? ('\r\n' + c.DDmp_MailingStreetLine2__c) : '') +
                       ((c.DDmp_MailingStreetLine3__c <> null) ? ('\r\n' + c.DDmp_MailingStreetLine3__c) : '');
                   if (c.MailingCity==null && c.DDmp_MailingCity__c<>null) c.MailingCity = c.DDmp_MailingCity__c;
                   if (c.MailingState==null && c.DDmp_MailingState__c<>null) c.MailingState = c.DDmp_MailingState__c;
                   if (c.MailingPostalCode==null && c.DDmp_MailingZip__c<>null) c.MailingPostalCode = c.DDmp_MailingZip__c;
                   if (c.MailingCountry==null && c.DDmp_MailingCountry__c<>null) c.MailingCountry = c.DDmp_MailingCountry__c;
                   // Take phone number from best in preference order Home, Mobile, Work
                   if (c.Phone==null) {
                       if (c.DDmp_HomePhone__c<>null) c.Phone=c.DDmp_HomePhone__c;
                       else if (c.DDmp_MobilePhone__c<>null) c.Phone=c.DDmp_MobilePhone__c;
                       else c.Phone=c.DDmp_Work_Phone__c;
                   }
                   // If another phone exists, take it from 2nd-best in preference order Home, Mobile, Work
                   if (c.OtherPhone == null) {
                       if (c.DDmp_HomePhone__c<>null) {c.OtherPhone = (c.DDmp_MobilePhone__c<>null) ? c.DDmp_MobilePhone__c : c.DDmp_Work_Phone__c ;}
                   }
               }
        }
    }
    ...
}

---

Updated question: Changing my approach to recursion prevention worked (see my Answer), but required an After trigger, which seems less efficient. Is there anything I can do to have a Before trigger with the working style of recursion prevention? (There's no ID to latch onto and cache...)

Answer 1

Okay, so the following helped:

1. I changed the trigger to an After trigger. It's got a bunch more overhead (e.g. re-querying the records), but I was desperate and just trying anything different.

2. In that After trigger, instead of doing the standard "Static Boolean"-based recursion prevention, I switched to "check the IDs you have already processed-based" recursion prevention as indicated here.

New code:

public with sharing class ContactTriggerHandler {
...
    public static void copyDailyDumpDataAfterInsertOrUpdate(Map<Id,Contact> newContacts, Map<Id, Contact> oldContacts, Boolean isInsert) {
        // Make sure this method only runs once per set of just-inserted/updated contacts.
        if (alreadyProcessed.containsAll(newContacts.keySet())) {return;}
        alreadyProcessed.addAll(newContacts.keySet());

        List<Id> dumpCIds = new List<Id>();
        List<Contact> csToUpdate = new List<Contact>();

        for (Contact c : newContacts.values()) {
            // Cache as an "DDmp" Contact record if it was just inserted/updated by the DDmp Jitterbit load
            if (c.DailyDump_Contact_Id__c <> null &&
                c.DDmp_Jitterbit_Most_Recent_Load_DateTime__c <> null &&
                (isInsert ||
                 oldContacts == null ||
                 oldContacts.get(c.Id).DDmp_Jitterbit_Most_Recent_Load_DateTime__c == null ||
                 oldContacts.get(c.Id).DDmp_Jitterbit_Most_Recent_Load_DateTime__c < c.DDmp_Jitterbit_Most_Recent_Load_DateTime__c)
               ) {
                   dumpCIds.add(c.Id);
               }
        }

        if (dumpCIds.size()>0) {
            Boolean chgdDump = false;
            for (Contact c : [SELECT Id
                              ,PIDM__c,DDmp_Copy_of_PIDM__c
                              ,Banner_UST_ID__c,DDmp_Copy_Of_Banner_ID__c
                              ,Birthdate,DDmp_Birthdate__c
                              ,Citizenship__c,DDmp_Applicant_Citizenship_Status__c
                              ,Company_Holding_Spot__c,DDmp_Account_Name__c
                              ,Email,DDmp_Email__c
                              ,FirstName,DDmp_FirstName__c
                              ,Gender__c,DDmp_Gender__c
                              ,MailingStreet,DDmp_MailingStreetLine1__c,DDmp_MailingStreetLine2__c,DDmp_MailingStreetLine3__c
                              ,MailingCity,DDmp_MailingCity__c
                              ,MailingState,DDmp_MailingState__c
                              ,MailingPostalCode,DDmp_MailingZip__c
                              ,MailingCountry,DDmp_MailingCountry__c
                              ,Phone,OtherPhone,DDmp_HomePhone__c,DDmp_MobilePhone__c,DDmp_Work_Phone__c
                              FROM Contact WHERE Id IN :dumpCIds]) {
                                  {
                                      // Smart-edit various fields
                                      if (c.PIDM__c==null && c.DDmp_Copy_of_PIDM__c<>null) c.PIDM__c = c.DDmp_Copy_of_PIDM__c; chgdDump=TRUE;
                                      if (c.Banner_UST_ID__c==null && c.DDmp_Copy_Of_Banner_ID__c<>null) c.Banner_UST_ID__c = c.DDmp_Copy_Of_Banner_ID__c; chgdDump=TRUE;
                                      if (c.Birthdate==null && c.DDmp_Birthdate__c<>null) c.Birthdate = c.DDmp_Birthdate__c; chgdDump=TRUE;
                                      if (c.Citizenship__c==null && c.DDmp_Applicant_Citizenship_Status__c<>null) c.Citizenship__c = c.DDmp_Applicant_Citizenship_Status__c; chgdDump=TRUE;
                                      if (c.Company_Holding_Spot__c==null && c.DDmp_Account_Name__c<>null) c.Company_Holding_Spot__c = c.DDmp_Account_Name__c; chgdDump=TRUE;
                                      if (c.Email==null && c.DDmp_Email__c<>null) c.Email = c.DDmp_Email__c; chgdDump=TRUE;
                                      System.debug('First names:  ' + c.FirstName + c.DDmp_FirstName__c); // DEBUG LINE ONLY
                                      if ((c.FirstName==null || Test.IsRunningTest()) && c.DDmp_FirstName__c<>null) c.FirstName = c.DDmp_FirstName__c; chgdDump=TRUE; // To make debug logs easier to read, our test to make sure this works starts with a FirstName in place, unlike real data, hence the alternative check for isRunningTest for overwriting.
                                      if (c.Gender__c==null && c.DDmp_Gender__c<>null) c.Gender__c = c.DDmp_Gender__c; chgdDump=TRUE;
                                      if (c.MailingStreet==null && c.DDmp_MailingStreetLine1__c<>null) c.MailingStreet = c.DDmp_MailingStreetLine1__c +
                                          ((c.DDmp_MailingStreetLine2__c <> null) ? ('\r\n' + c.DDmp_MailingStreetLine2__c) : '') +
                                          ((c.DDmp_MailingStreetLine3__c <> null) ? ('\r\n' + c.DDmp_MailingStreetLine3__c) : ''); chgdDump=TRUE;
                                      if (c.MailingCity==null && c.DDmp_MailingCity__c<>null) c.MailingCity = c.DDmp_MailingCity__c; chgdDump=TRUE;
                                      if (c.MailingState==null && c.DDmp_MailingState__c<>null) c.MailingState = c.DDmp_MailingState__c; chgdDump=TRUE;
                                      if (c.MailingPostalCode==null && c.DDmp_MailingZip__c<>null) c.MailingPostalCode = c.DDmp_MailingZip__c; chgdDump=TRUE;
                                      if (c.MailingCountry==null && c.DDmp_MailingCountry__c<>null) c.MailingCountry = c.DDmp_MailingCountry__c; chgdDump=TRUE;
                                      // Take phone number from best in preference order Home, Mobile, Work
                                      if (c.Phone==null) {
                                          if (c.DDmp_HomePhone__c<>null) {c.Phone=c.DDmp_HomePhone__c; chgdDump=TRUE;}
                                          else if (c.DDmp_MobilePhone__c<>null) {c.Phone=c.DDmp_MobilePhone__c; chgdDump=TRUE;}
                                          else c.Phone=c.DDmp_Work_Phone__c; chgdDump=TRUE;
                                      }
                                      // If another phone exists, take it from 2nd-best in preference order Home, Mobile, Work
                                      if (c.OtherPhone == null) {
                                          if (c.DDmp_HomePhone__c<>null) {c.OtherPhone = (c.DDmp_MobilePhone__c<>null) ? c.DDmp_MobilePhone__c : c.DDmp_Work_Phone__c ; chgdDump=TRUE;}
                                      }
                                  }
                                  // If anything was smart-edited, set the record aside for updating
                                  if (chgdDump) csToUpdate.add(c);
                                  chgdDump=FALSE;
                              }
        }

        if(csToUpdate.size()>0) {
            UPDATE csToUpdate;
        } 
    }
    ...
}

I'm glad it's working, but for future reference, does anyone have any suggestions how I could achieve the same "large dataset recursion prevention" effect with a Before trigger in this context? (There's no "Id" to latch onto and cache at the class level yet...)