0

For a synchronisation I need to import a somewhat large dataset from a REST API. There around 30k records and the API I consume supports pagination. What would be the best approach to retrieve these records and not kill the API with 1000 parallel requests. I've looked at batching, but that depends on SF records. I've looked at Chaining Asynchronous Callouts but that only works for VF pages (if it works)

Is there a mechanisme for this I can use?

  • Batch job can be run on strings, not only on Salesforce objects. If your api URL is like <some.url/route>/take=<amount>&skip=<amount>, then you can create array of those, and start batch on them – kurunve Mar 6 '18 at 11:34
  • Yes, I thought of something like that, but in practice the batches would still fire almost simultaneous, so it wouldn't be different from future calls. – Marco Pietersen Mar 6 '18 at 11:56
  • Batches will run one by one, not in parallel – kurunve Mar 6 '18 at 11:58
  • @kurunve you are right; but I opted for queue's in the end. – Marco Pietersen Mar 7 '18 at 15:16
1

The best option for me turned out to be chaining Queueable Jobs. There are no limits on how many recursions you can do so that suited me.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.