Polling endpoint in Apex

Question

I have a callout to an API that responds with a job number. With that job number as a parameter I keep polling an endpoint, and basically receive a PENDING response until their server compiles an NDJSON file, at which point I get a SUCCESS response (and a url to the file).

I've been researching how to poll the API in apex, as it could take anywhere from sub one second to about a minute. A response to this post refers to the Continuation class and API Streaming but I've been going through documentation and examples and neither seem to be a good fit (re: Continuation - it's not a long wait for a response necessarily, I'm waiting for a specific response, and it doesn't look configurable for that - re: API Streaming - I'm the client in this situation, wouldn't this need to be initiated by the server?)

The other option mentioned is using a setTimeout function in JavaScript, but wondering if there's an apex approach I'm missing. Thanks for any help.

Answer 1

Since this may be completely asynchronous, it sounds like you generally want to use a Queueable class. Your your queueable class would spin for a while, make the callout, then re-queue itself if necessary. That would look something like this:

public class PollWait implements Queueable {
  public void execute(QueueableContext context) {
    Long startTime = DateTime.now().getTime();
    // Basically, Thread.sleep, because we can't sleep.
    // The ; at the end indicates an empty while body.
    // This is "busy waiting", since we're using CPU time doing "nothing".
    while(DateTime.now().getTime()-startTime<1000); 
    CalloutResponse res = CalloutUtil.doCallout(...); // this is pseudo-code
    if(res.isStillProcessing) {
      System.enqueueJob(this);
    } else {
      // handle the result data
    }
  }
}

You'll need to add whatever properties you need to track, etc, but this is the basic idea you'd want to use. Adjust the spin loop to use as much or as little time as you think would be appropriate. You have up to 60,000ms of CPU time you can spin for.

Note that the "empty body" while loop is intentional. Document this carefully so you won't forget that it's meant to do this.

You could actually use the busy wait for up to 60,000ms, so you could also put this in a loop to have the best chance of only needing a single asynchronous call (we're limited to 250,000 calls per day).