Is it guaranteed that First submitted Queueable or Batch class will run first?

Question

We have to send case create and update data via API to third party.

For create case: In trigger we are collecting all the case ids in set and calling queueable class if cases created are less than 100 and Batch class if cases created are more than 100.

For update case: We have some custom object where we are tracking specific fields changes with date/time with every dml happening from user or any async job or scheduled job. The custom object records we are sending as part of Scheduled job running every 5 minutes.

After doing a lot of testing the only grey area i feel is ::: If a case is created, queueable or batch class is fired first. On UI, Suppose user change one field which we are tracking within 5 seconds of case creation from UI. By chance the scheduled job ran that time just after this which is calling another batch class to send data to third party for UPDATES only.

Question : Do you feel that I could land up where on creation, batch or queueable which is fired first does not get executed earlier than the batch which used to send data as part of update case field as both are getting executed?

Answer 1

Given that there are no documented guarantees around order of execution for Queueable implementations (these do not go into a queue in the same way that Batchables do), I would consider creating a "single threaded" processing approach instead. By ensuring that all the outgoing messages are handled through a mechanism designed to ensure that there is no (or using specifically controlled) concurrency, and basing order of send on the created date for your custom records, you can guarantee the correct order of delivery for deltas applied to any given Case record.

It is easy enough to create a "single threaded" Batchable (this is something we did for ourselves several years ago). I haven't, as yet, created a general purpose "single threaded" Queueable. However, in this case I believe that is actually unnecessary.

I have created an example of an approach based on a "command object", Platform Events and a simple callout handling Queueable - I'd have loved to have avoided the latter, but Salesforce does not (yet) support making a callout directly from a Platform Event subscriber.

The principles here are:

1. To capture creation, update and delete events for Case (in this specific example) in a custom object, here called Command__c. This includes a status to help control ordering of sending for any given record and a JSON string of the relevant fields and their values.
2. To generate a Platform Event that causes the out-of-transaction processing of these "commands". This avoids any nasty governor limits for initiating async Apex within triggers, regardless of whether the trigger is handling bulk updates and/or is itself being called from an async context.
3. To use a simple Queueable to convert up to 100 commands per execution into callouts, while ensuring that the order of "commands" is retained for any given record.

You can find code demonstrating this in github - the only thing it doesn't do is make any callouts (but you can find where this should be done by searching for "TODO").

Some of this code is lifted from another of my repos, one supporting an article I wrote for ApexHours.

NB: While I've had a play with this with some individual and bulk DML operations (create, update and delete), I do not actually guarantee that it really is doing what I've said and ensuring that send order is maintained - I will do some specific analysis by adding debug in the near future and update the repo if required.

UPDATE: The following are Q&As related to the example solution based on questions added to this answer.

Where in the process should I capture the data to send?

This needs to be captured in the DML processing for the record (here Case) insert/update/delete.

To try to capture this asynchronously (e.g. in the platform event trigger handler) means you could end up querying the case(s) being created or updated after they have already been further updated (or even deleted), meaning you would not capture full history.

Additionally, to do this capture outside the DML context means you won't know which fields changed and cannot easily compute the delta to only send individual field changes when handling DML updates.

The example code captures the field values into the Command records within the context of the Case trigger.

Yes, this can introduce CPU limit issues if someone tries to bulk insert or bulk update a large number of Cases - with this code in place I got a CPU timeout when trying to bulk insert and update 1000 cases in a single transaction (in two DMLs), though I could successfully bulk insert and update 500 cases together.

Won't creating a "Command" per DML cause other governor limit issues?

If there is a lot of automation against Case (or other target object), or the calling code is trying to perform massive bulk insert/update/delete operations against the object, it is possible to hit various governor limits (e.g. DML row issues or CPU timeout as mentioned above). However, in the callout side of things, this sequential generation of "Commands" should not directly cause governor limit issues.

Once the Commands exist in the database, the platform event trigger invokes a processor that in turn queries Commands waiting to be processed (using oldest to newest and ensuring that no Commands are processed if there is already one on-going for a given record). This ensures that at most 100 Commands are processed at a time, thus staying within transaction callout limits, leaving others to the next time the platform event trigger is called (and ensuring this will happen by raising a new platform event).

If 100 is too many, e.g. because the automation for the callout is too complex and you would hit CPU limits or cumulative callout timeouts, you can simply reduce the number of Commands processed together by changing the LIMIT on the SOQL in the CommandProcessor.

This will send callouts when a change X is made that is then reversed out

Indeed, if you have a Case where the description (or some other field) with value A is changed to value B then back to A in two separate DML operations, this will generate callouts. However, this is the correct behaviour because for a short while the Case's description (or other field) did change value.

To try to "optimize" this makes little sense to me; you have no idea how long the time is between the first and second change.

The one optimization that is worthwhile (and that I didn't do) is to detect that no field values changed and avoid generating a Command in the first place.

What happens if the end point has downtime?

In this case we just need to keep the Commands and use them again for later retries.

I didn't show any error handling in the CommandSender, but that is easy to add. The key is for the sender to capture the set of Command IDs for which sending failed into a new Set of IDs and at the end of its execute ensure it doesn't update these Commands to Complete status. Instead it should update its own commandIds set to reflect this content then explicitly re-enqueue itself via System.enqueueJob, but this time explicitly passing a delay. This then easily implements a retry.

You could even implement a back-off process, lengthening the delay after each re-try, by also storing an attribute in the queueable that is the previous delay value (defaulting to 0 to indicate there was no delay before the first re-try) and using this to calculate the next delay value.

You may need to ensure that you don't enqueue further CommandSender Queueables in the CommandProcessor during this downtime (I leave this as an exercise for the reader, but I'd consider having a "Resending" status for a Command).

Your Command cannot cope with Long Text Area fields

True indeed. Here I would treat those fields specially, having specific additional "generic" Long Text Area fields in the Command that are explicitly used to store Long Text Area field values (for "known" fields of this type for a given object) and update the UpdatedFieldsJSON__c generation to skip these fields; once the JSON is generated I would iterate the required Long Text Area fields and (conditionally, for update) process them through to these separate fields.

Obviously you may need to explicitly configure some mapping for these, or you may explicitly store the name of the long text area field being stored, alongside the value, using two separate fields... e.g.:

• LongTextAreaFieldName1__c - to store the field name.
• LongTextAreaFieldValue1__c - to store the field value.

You could have n of these field pairs you simply dynamically iterate to store insert/update for these Long Text Areas. Obviously if there are more than the n fields in your Case (or other object) you will need to add extra pairs into the Command.

Can we store history of the updates?

Yes, in fact that is exactly what you get with the Command. These describe the changes applied (and could capture who did it too, if you wanted). It's just that the Command also includes a Status that controls how they are converted to callouts.

Now, directly storing all history in the org is not going to scale. Leaving the Commands in the database will use a lot of storage and, worse, cause SOQL scalability/selectivity issues down the line.

You should only keep history for the recent past in the org, optionally archiving and explicitly deleting Commands when they are old enough (and in Complete status).

Couldn't this be easier to implement?

Well, yes, it could, if Salesforce supported callouts from Platform Event trigger subscribers.

If this was supported, we get a simple way to implement single-threaded handling for all the callouts. The down-sides would be:

1. You cannot recover from governor limit issues (which you can using a Finalizer with a Queueable).
2. It would be literally single-threaded. There would be no sensible way to do parallel sending of independent Commands (i.e. relating to different records) without re-introducing the Queueable.

Feel free to upvote this idea for direct callout support.

Answer 2

Phil, please review:

I was thinking as platform event cant actually handle more than 1 mb (Long text area fields) and we are not storing any data if somehow the transaction doesn't succeed. Below could be the idea that will work:

1. Created a custom object with name as Transaction History.

2. IsSynced__c -- It will be used by batch class to send data in sequence using Queueable class passing 2000 records at once to batch once with order by Transaction_Date_Time__c. Once the data is sent successfully,IsSynced__c should be marked as true.

3. Foreign_key__c will actually play a major role in saving one SOQL within trigger and also identifying whatever the transaction history record has already been created for multiple scenarios where a case got created and updated within a same transaction.

4. Now a below code in CaseTriggerhandler will be added to create transaction history record which will tell which case was modified by whom and when.

5. For each transaction history record, multiple files can get created. Reason: If a case is created and updated 2 times within same transaction. Or If a case is updated, then via code one new case is created or updated within same transaction. Files will have all the fields of Trigger.new case record. So if 3 times one case is updated, 3 files will be created under one transaction history record. The latest record of that transaction will tell what were the final values of the case record.

6. Run a batch class - Using queueable to send data in sequence to third party (As there is no callout limit). In Queueable we will query the records where IsSynced__c = False And sObject = 'Case' Order By Transaction_Date_Time__c

7. This means for we will send whole Case record from salesforce to third party and let the third party see the difference between fields (or update it directly)

public without sharing class CaseTriggerHandler extends TriggerHandler {

//Will be used for Creating Transaction History Records - START
public Static Map<String,Id> keyToTransactionIdMap = new Map<String,Id>();
public Static String currentRequestId = '';
public Static DateTime currentDateTime;
//Will be used for Creating Transaction History Records - END
    

static {
    currentRequestId = Request.getCurrent().getRequestId();     //Get the transaction ID - Used for creating a unique key
    currentDateTime = System.now();                             //Hold the first datetime value when actually transaction kicks in
}


/* Description: After insert and After update method to capture the history of the case on insert and update
 * Modification: 
*/
public void updateTransactionHistory(){
    Map<String,String> transHistoryKeyToCaseStrMap = new Map<String,String>();
    Map<String,ContentVersion> transKeyToCVMap = new Map<String,ContentVersion>();
    
    List<Transaction_History__c> transHistoryInsertList = new List<Transaction_History__c>();
    List<ContentDocumentLink> cdlList = new List<ContentDocumentLink>();
    
    for (Case caseObj : newCaseList){

        if(Trigger.isAfter && (Trigger.isInsert || Trigger.isUpdate)){
            
            String foreignKeyString = currentRequestId+String.valueOf(currentDateTime)+'**'+caseObj.Id+'**';
            foreignKeyString = foreignKeyString.deleteWhitespace();
            
            //Below code will not allow to create another transaction history if one has already been created.

            if(!keyToTransactionIdMap.containskey(foreignKeyString)){
                Transaction_History__c transactionHistoryObj = new Transaction_History__c(Foreign_Key__c = foreignKeyString, Transaction_Date_Time__c = currentDateTime, sObject__c = 'Case', Case__c = caseObj.Id, User__c = UserInfo.getUserId(), Operation__c = Trigger.isInsert ? 'CreateCase' : 'UpdateCase');
                transHistoryInsertList.add(transactionHistoryObj);  
            }
            
            //Serialize the whole object of case - START
            Map<String, Object> caseMapDeserialized = (Map<String, Object>) JSON.deserializeUntyped(JSON.serialize(caseObj)) ;
            caseMapDeserialized.remove( 'attributes' );
        
            transHistoryKeyToCaseStrMap.put(foreignKeyString,JSON.serialize(caseMapDeserialized));
            //Serialize the whole object of case - END
            
        }
    }
    
    if(!transHistoryInsertList.isEmpty()){
        insert transHistoryInsertList;
        
        for(Transaction_History__c transHistoryObj : transHistoryInsertList){
            keyToTransactionIdMap.put(transHistoryObj.Foreign_Key__c, transHistoryObj.Id);
        }
    }
    
    if(!transHistoryKeyToCaseStrMap.isEmpty()){
        
        for(String key : transHistoryKeyToCaseStrMap.keyset()){
            ContentVersion cvObj = new ContentVersion();
            cvObj.Title = 'FieldsChanged';
            cvObj.PathOnClient = 'FieldsChanged.txt';
            cvObj.VersionData = Blob.valueOf(transHistoryKeyToCaseStrMap.get(key));
            cvObj.FirstPublishLocationId  = keyToTransactionIdMap.get(key);
            transKeyToCVMap.put(key,cvObj);
        }
        
        if(!transKeyToCVMap.isEmpty()){
            insert transKeyToCVMap.values();
        }
    
    }   
}

}