0

For a synchronisation I need to import a somewhat large dataset from a REST API. There around 30k records and the API I consume supports pagination. What would be the best approach to retrieve these records and not kill the API with 1000 parallel requests. I've looked at batching, but that depends on SF records. I've looked at Chaining Asynchronous Callouts but that only works for VF pages (if it works)

Is there a mechanisme for this I can use?

Improve this question
4
  • Batch job can be run on strings, not only on Salesforce objects. If your api URL is like <some.url/route>/take=<amount>&skip=<amount>, then you can create array of those, and start batch on them
    – kurunve
    Commented Mar 6, 2018 at 11:34
  • Yes, I thought of something like that, but in practice the batches would still fire almost simultaneous, so it wouldn't be different from future calls.
    – Marco Pietersen
    Commented Mar 6, 2018 at 11:56
  • Batches will run one by one, not in parallel
    – kurunve
    Commented Mar 6, 2018 at 11:58
  • @kurunve you are right; but I opted for queue's in the end.
    – Marco Pietersen
    Commented Mar 7, 2018 at 15:16

1 Answer 1

Reset to default
1

The best option for me turned out to be chaining Queueable Jobs. There are no limits on how many recursions you can do so that suited me.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .