Refactoring Apex synchronous callout to asynchronous

Question

I'm refactoring a Visualforce page, which currently gets data from a synchronous Apex callout, so that the callout will asynchronous. The Visualforce page downloads a .csv file with customer data in it. In some instances the customer data exceeds the 6MB limit for synchronous Apex callouts. Asynchronous calls have a 12MB limit, so the technical direction given is to utilize that with Continuation.

But I'm running into an issue where the Visualforce page is not returning the data after having made the switch to use an asynchronous call. I'm following the steps Salesforce has documented for making long-running callouts with Continuation. Here's a breakdown:

I have a CommonController class that calls an endpoint. That class is called by a APIUtils class, which gets called by a HistoryController class; HistoryController is called by the Visualforce page.

CommonController.cls:

The CommonController class makes an HttpRequest to an endpoint using Continuation.

public static string methodStr;
@AuraEnabled(continuation=true)
public static Object invokeAPIContinuation(String servicePath, String httpMethod, 
String httpBody,Map<String,String> metaDataMap){
    String serviceEndPoint;
    Continuation con = new Continuation(40);
    con.ContinuationMethod = 'conResponseHandler';
    
    // the rest of this method is unchanged from the synchronous way...
    serviceEndPoint = metaDataMap.get('APIEndpoint'); 
    String apiEndPoint= serviceEndPoint + servicePath;
    HttpRequest httpRequest = new HttpRequest();  
    httpRequest.setMethod(httpMethod);   
    httpRequest.setHeader('Content-Type', 'application/json');
    String crtNo = metaDataMap.get('APICertName'); 
    httpRequest.setClientCertificateName(crtNo);
    httpRequest.setEndpoint(apiEndPoint); 
    if(httpMethod.endsWithIgnoreCase('POST')) {
        httpRequest.setBody(httpBody);
        methodStr = 'POST';
    }

    Integer timeOut = Integer.valueof(metaDataMap.get('TimeOut'));
    httpRequest.setTimeout(timeOut); 
    con.addHttpRequest(httpRequest);
    return con;
}  

// make a callback method as outlined in the Salesforce documentation for Continuation...
public static String conResponseHandler(List<String> labels, Object state){
    // Get the response by using the unique label
    HttpResponse httpResponse = Continuation.getResponse(labels[0]);
    // Set the result variable
    String responseBody = httpResponse.getBody();
    String response = '';
  
    if(httpResponse.getStatusCode() == 200 ) { 
        responseBody = httpResponse.getBody();
        if(methodStr == 'POST') {
            responseBody = 'success';
        }
    }
    return responseBody;
}

APIUtils.cls

The APIUtils class calls invokeAPIContinuation from the CommonController class.

public static void getEmployeeDetailsDataCont(String billingNumber, String enterpriseId, String billingControlNumber, Map<String,String> metaDataMap,String benefitType) {
    String servicePath = billingNumber +'/Members?EmployerId=' + enterpriseId + '&BenefitType=' + benefitType;
    CommonController.invokeAPIContinuation(servicePath, 'GET', '',metaDataMap);
}

HistoryController.cls:

The HistoryController class calls getEmployeeDetailsDataCont from the APIUtils class.

public void exportEmployeeDataToCsvFile() {
    String empTypeParam = ApexPages.currentPage().getParameters().get('empType');
    if(empTypeParam.equalsIgnoreCase('GB')) {
        contentType = 'text/vnd.ms-excel#groupDetails.csv';
    }else {
        contentType = 'text/vnd.ms-excel#voluntaryDetails.csv';
    }
    String billingNumber = ApexPages.currentPage().getParameters().get('billingNumber');
    Map<String,String> metaDataMap = CommonController.getmetaDataValues();// = getBillingMetaDataForVFPage();

    // comment out previous method to use the new Continuation method
    // csvFileEmployeeData = APIUtils.getEmployeeDetailsData(billingNumber,CommonController.getEmployerIdFromLoggedInUser(),'',metaDataMap,empTypeParam);
    APIUtils.getEmployeeDetailsDataCont(billingNumber,CommonController.getEmployerIdFromLoggedInUser(),'',metaDataMap,empTypeParam);
}

Visualforce page:

<apex:page controller="HistoryController" contentType="{!contentType}" action="{!exportEmployeeDataToCsvFile}">{!csvFileEmployeeData}</apex:page>

But my Visualforce page is not returning data after the asynchronous call. I suspect this may have partially to do with the APIUtils class method previously returned a string but now returns a void. At this point I'm pretty stumped.

Answer 1

Disclaimer: I've never used continuations; I just read about them earlier this year. According to comments on the question, it sounds like that feature is old and on the path to deprecation.

I have used Async apex a lot,of the variety that gives you 60 seconds of execution time rather than 10 -- namely batch apex, future methods, and my personal favorite, queueables. At this point, I never use batch apex or future methods for new features, as my company's use cases and user requests are too complex to fit into that mold.

Queueables offer a lot of benefits in terms of flexibility (they can pass any serializable data from one execution to the next), but reporting queueable execution to a UI can be challenging if your use case chains queueables to run one after another.

Our company has therefore found it useful to bind our asynchronous processing to records in the system that store the result of that processing. This allows us to run reports off of the results, and build UIs around them. For your use case, I would recommend the following:

1. Create a new object to represent the CSV generation request. Give it a status field with the values "Running, Queued, Complete, Failed" or something like that. Give it a long text area field for storing error messages. Give it a text field for storing the queueable job id. Give it a URL field for storing a link to the CSV attachment when it's created.
2. Create a trigger handler for that object. When the status is set to "Running" kick off a Queueable job that performs the callout. Store the job id in the record that is running. Bulkifying this piece may require some additional elegance, since there's a limit to the number of queueables you can enqueue per transaction. but for the sake of discussion, I'll ignore that issue.
3. The queueable class should have a property to store the id of the record to which it corresponds. In the execute method, first create a try/catch statement. If any error occurs, have the system update the record with the error message, and update the status to "Failed".
4. Within the try/catch statement, perform the callout. If the callout yields an error, throw an exception. Otherwise, attach the CSV result as a file attachment on the record. Update the record to status "Complete", and update the Link field to link to the newly created attachment.

Next, update the UI in the following ways:

1. If possible, convert the visualforce page to a lightning component, or mount a lightning component on the VF page. This will give you more options.
2. In your VF / lightning controller write a method to insert a new record from your new object. Status should be set to "Running".
3. In your VF / lightning controller, write a method to query a record with the provided id, together with the status of its queueable job. If the queueable is failed, update the record with a "Failed" status and the queueable error message.
4. In your VF / lightning component, write javascript to call the method that creates a record of the new type. Poll the server every N seconds to retrieve its current status. When the data on the current record is returned, check if its status is "Failed", and if so, display an error message. Otherwise, check if the status is "Complete", and if so display a link to the download.

In #3 above, checking the job status will allow you to capture hard errors like CPU timeouts, memory limits, and other hard errors, prior to returning the record. If you don't do this, the queueable can fail without recording an error, because limit exceptions cannot be caught and handled.