I need to make an Apex Callout from salesforce to 3rd party website. This process needs to run once or twice a day. As this is not real time integration, and data can be large, so decided to go for Asynchronous process. Once we receive data, it needs to be saved into a log object and then process the data, to be saved into some other relevant objects.

Following are the approaches which I can think of for now -

  1. Write one batch class
    • through which we make an API Callout via constructor() and then perform the DML operation to save the incoming records into log object. Yes we can make an API callout and perform the DML from constructor, but know that it is not recommended.
    • write the query to get the new records from log object in start() method.
    • perform the logic on records and save it to relevant objects in the execute() method.
  2. Write two batch class
    • In first batch
      • we make an API callout from execute() method and save the records into log object.
      • as a query (or and iterable list) is mandatory for start() method, we make a dummy list here which will not be used anywhere.
      • then from the finish() method we call other batch class.
    • In second batch
      • we query the log objects to get the records for processing from the start() method.
      • we perform the logic inside execute() method and save the records in relevant objects.
  3. write one queueable class and one batch class
    • In first queueable class
      • perform the API callout in execute() method, and save the records into log object.
      • call the batch class from here.
    • In second batch class
      • get the records from log object into start() method.
      • perform the logic and save the records into relevant objects in execute() method.

I do not want to go against any design principal, so please recommend me the best way to achieve this.

Also I need to schedule the batch process

  • How about using a Schedulable to perform the callout and apply an "adaptive batch" to process the log conversion logic?
    – Phil W
    Jan 15 at 9:15
  • @PhilW I did not get your idea bout using adaptive batch exactly. Could you please explain a bit more in my scenario, with some code hint. Jan 15 at 9:27
  • The adaptive batch ensures data to be processed is processed uniquely, supporting re-trying or not and deliberately running in the Automated Process user context. It is a batch, but ensures only one instance ever processes data at a time, allowing one batch to process the full scope of data.
    – Phil W
    Jan 15 at 9:51
  • NB: This community isn't a free coding service and we require the asker to show what they have tried and explain where they are stuck. This question is likely to get closed without answer unless you edit it to meet community requirements.
    – Phil W
    Jan 15 at 9:53
  • I have tried explaining each and everything here, which I know about. I didn't asked the question bluntly. I have done my research and that's why have mentioned limitations and outcome for each process. I didn't expect a guy of your stature should comment like this - demotivating newcomer. Jan 15 at 10:18